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Abstract 

Earned  Schedule,  since  it  was  introduced  by  Walt  Lipke  in  2003,  has  been  studied 
extensively  in  a  variety  of  different  fields  and  on  programs  of  all  sizes.  However,  Earned 
Schedule’s  viability  as  an  extension  of  Earned  Value  Management  (EVM)  in  Major 
Defense  Acquisition  Programs  (MDAP)  has  yet  to  be  effectively  answered.  The  first 
aspect  of  this  research  explores  the  breadth  of  Earned  Schedule’s  adoption  by  the  System 
Program  Offices  (SPO)  of  the  United  States  Air  Force.  The  second  phase  of  this  research 
explores  whether  Earned  Schedule  is  a  more  accurate  and  timely  schedule  predictor  than 
the  EVM  technique  currently  employed  by  the  United  States  Department  of  Defense 
(DoD).  A  series  of  five  descriptive  statistical  tests  were  conducted  on  the  Earned  Value 
data  for  64  Acquisition  Category  (ACAT)  I  MDAP’s.  This  research  finds  Earned 
Schedule  to  be  a  more  timely  and  accurate  predictor  than  Earned  Value  Management. 
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A  Comparison  of  Earned  Value  Management  and  Earned  Schedule  as  Schedule 
Predictors  on  DoD  ACAT  I  Programs 


I.  Introduction 


General  Issue 

One  would  be  hard  pressed  to  open  the  Wall  Street  Journal  or  the  New  York 
Times  on  any  given  day  without  encountering  a  number  of  articles  related  to  the  broad 
financial  cuts  the  Department  of  Defense  (DoD)  will  experience  over  the  next  decade.  In 
the  financially  lean  environment  under  which  the  DoD  now  operates,  effective 
management  of  a  program’s  cost  and  schedule  performance  has  never  been  more  vital. 
However,  the  DoD  has  long  struggled  with  cost  overruns  and  schedule  delays  on  major 
acquisition  projects.  Earned  Value  Management  (EVM)  has  been  the  premier  method  of 
program  management  and  program  cost  forecasting  within  the  DoD  since  its  inception  in 
the  1960s.  EVM  has  long  been  hailed  for  its  ability  to  identify  to  the  decision-maker 
whether  a  program  is  going  to  be  over  cost  or  over  schedule.  However,  EVMS’s  merit  of 
forecasting  schedule  overages  has  been  questioned  in  recent  years  (Lipke,  2003:  1).  The 
predominant  shortcoming  of  EVM  is  how  it  measures  schedule  performance:  it  quantifies 
schedule  overages  in  terms  of  dollars  ($),  rather  than  in  terms  of  time.  This  means  of 
measurement  is  ambiguous  and  potentially  confusing  to  program  managers.  To  overcome 
this  problem,  a  new  schedule  measurement  technique,  Earned  Schedule  (ES)  was 
developed  (Lipke,  2003:  1).  Earned  Schedule  rectifies  the  ambiguities  of  traditional 
EVMS  schedule  analysis  by  expressing  schedule  measurements  in  terms  of  time.  It  has 
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been  argued  that  the  critical  development  of  ES  provides  program  managers  the 
predictive  tool  needed  to  determine  project  completion  dates  using  EVM  data. 

Background 

While  the  origins  of  Earned  Value  Management  can  be  traced  to  the  factory  floor 
during  the  industrial  revolution  of  the  1800s,  it  was  introduced  in  its  current  form  to  the 
agencies  of  the  U.S.  government  in  1967  as  the  Cost/Schedule  Control  Systems  Criteria 
(C/SCSC).  EVM  has  been  predominantly  used  in  program  management  within  the 
federal  government,  but  its  use  in  the  private  sector  has  grown  exponentially  in  the  last  20 
years.  EVM  has  been  widely  lauded  for  its  ability  to  improve  cost,  schedule  and 
technical  performance  of  a  project;  however,  its  limitations  in  its  ability  to  predict 
schedule  overruns  are  widely  recognized.  Use  of  EVM  in  commercial  industry  under  the 
C/SCSC  was  nearly  non-existent  due  to  the  overheads  imposed  by  stiff  regulations,  but 
its  use  increased  when  the  C/SCSC  regulations  were  dropped  in  favor  of  the  more 
flexible  Earned  Value  Management  System  (EVMS)  (Anbari,  2003:  12). 

There  are  three  parameters  that  are  the  basis  upon  which  traditional  earned 
value  analysis  is  built.  The  first  of  these  parameters  is  the  Budgeted  Cost  of  Work 
Scheduled  (BCWS),  also  known  as  the  Planned  Value  (PV)  in  commercial  industry. 
BCWS  is  a  “time-phased  budget  baseline  . . .  that  can  be  viewed  as  the  value  to  be  earned 
as  a  function  of  project  work  accomplishments  up  to  a  given  point  in  time”  (Anbari, 

2003:  13).  More  informally,  it  is  the  amount  of  work,  measured  in  dollars,  planned  for 
completion  during  a  measured  period  of  time.  The  second  parameter  is  the  Budgeted 
Cost  of  Work  Performed  (BCWP),  also  known  as  the  Earned  Value  (EV)  in  commercial 
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industry.  BCWP  is  the  “amount  budgeted  for  performing  the  work  that  was  accomplished 
by  a  given  point  in  time”  (Anbari,  2003:  13).  More  simply,  BCWP  is  the  value  of  the 
work  that  has  been  completed.  The  final  EVMS  parameter  is  the  Actual  Cost  of  Work 
Performed  (ACWP),  known  simply  as  Actual  Cost  (AC)  in  industry.  ACWP  is  the  cost 
in  dollars  of  all  the  work  that  has  been  performed  on  a  project  (Anbari,  2003:  13).  To 
more  clearly  illustrate  these  EVMS  concepts,  Table  1  gives  an  example  of  two  lines  of  a 
standard  Work  Breakdown  Structure  (WBS).  For  WBS  Package  1.1,  the  BCWS  is 
$100K,  while  only  $85K  worth  of  work  has  actually  been  completed.  Since  the  ACWP  is 
$95K,  costs  have  overrun.  Additionally,  because  the  value  of  work  performed  is  less 
than  the  amount  of  work  that  has  been  scheduled  for  completion,  this  project  is  also 
behind  schedule.  For  WBS  Package  1.2,  the  BCWS  is  equal  to  the  BCWP,  indicating 
that  all  work  has  been  completed  on  time.  With  the  ACWP  coming  in  below  the  value  of 
the  work  performed,  this  project  has  been  completed  not  only  on  time,  but  also  under 
budget. 


Table  1.  EVMS  Parameter  Example 


BCWS  ($K) 

BCWP  ($K) 

ACWP  ($K) 

WBS  Package  1.1 

100 

85 

95 

WBS  Package  1.2 

50 

50 

48 

Program  managers  utilize  two  metrics  in  the  traditional  EVMS  system  to 
ascertain  the  status  of  a  project’s  schedule.  These  traditional  schedule  specific  metrics 
used  under  the  umbrella  of  EVMS  are  Schedule  Variance  (SV($))  and  the  Schedule 
Performance  Index  in  terms  of  dollars  (SPI[$]).  Schedule  variance  tells  us  how  late  our 
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project  is,  but  does  so  in  terms  of  dollars.  If  a  project  is  $100K  late,  how  late  is  it  in  terms 
of  days,  months,  or  years?  SV($)  can’t  give  us  this  answer,  and  for  this  reason, 
proponents  of  Earned  Schedule  have  been  sharply  critical  of  it.  The  second  metric  used 
to  measure  schedule  under  EVMS  is  the  Schedule  Performance  Index  ($).  SPI($), 
according  to  Coro  vie,  is  “a  ratio  between  [BCWP]  and  [BCWS]  at  the  end  of  the  project, 
if  the  project  has  delivered  all  what  was  planned,  [BCWP]  and  [BCWS]  must  be  equal” 
(Coro vie,  2006-2007:  2)  Since  this  ratio  always  regresses  to  1.0,  there  is  little  value  in 
this  ratio  after  a  project  reaches  a  certain  point.  Analysis  has  shown  that  this  point  is  at 
about  2/3  completion  of  the  project  (Lipke,  2003:  1). 

Earned  Schedule  was  developed  in  2003  by  Walt  Lipke  in  direct  response 
to  the  aforementioned  EYMS  schedule  calculation  shortcomings.  Earned  Schedule  uses 
the  same  data  collected  for  the  earned  value  calculations,  but  focuses  on  giving  the 
program  manager  more  useful  information:  schedule  analysis  metrics  delivered  in  terms 
of  time,  rather  than  dollars,  as  well  as  the  ability  to  predict  if,  and  when,  a  project  may  go 
over  schedule. 

Problem  Statement 

The  current  EVMS  metrics  used  for  schedule  are  not  effective  predictors  over  the 
life  of  a  program  (Lipke,  2003:  1).  Earned  Schedule  (ES)  has  been  developed,  and 
consequently  studied  significantly  over  the  last  decade,  but  a  thorough  and  conclusive 
application  to  Acquisition  Category  I  (ACAT  I)  Major  Defense  Acquisition  Programs 
(MDAP)  has  yet  to  be  accomplished.  This  paper  will  research  whether  ES  is  a  more 
accurate  predictor  of  schedule  overruns  than  the  current  metrics  used,  whether  it  can 
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predict  schedule  overruns  earlier  in  the  life  of  a  program  than  the  current  schedule 
metrics,  and  how  accurately  ES  can  predict  the  final  completion  of  a  project  on  ACAT  I 
DoD  acquisition  programs. 

Methodology 

This  research  will  include  a  comprehensive  literature  review  of  all  relevant  works 
in  the  fields  of  earned  value  management,  earned  schedule,  and  their  application  to 
program  management.  Quantitatively,  a  comparison  of  means  (paired  t-test)  will  be 
conducted  on  the  contracts  for  64  different  ACAT  I  acquisition  programs  to  determine  if 
there  is  a  statistical  difference  between  Earned  Schedule  and  Earned  Value  Management. 
If  there  is  a  conclusive  difference  between  the  two  techniques,  a  series  of  tests  will  be 
conducted  on  the  data  to  determine  which  schedule  analysis  method  offers  more  valuable 
information  to  the  program  manager.  This  analysis  will  include  the  current  metrics  used 
under  the  umbrella  of  Earned  Value  Management  compared  to  the  new  metrics  derived 
under  Earned  Schedule. 

Research  Questions 

The  following  research  questions  will  be  investigated: 

1. )  To  what  extent  is  Earned  Schedule  currently  utilized  in  Air  Force  ACAT  I 

acquisition  programs? 

2. )  Does  Earned  Schedule  provide  more  accurate  schedule  predictions  than 

traditional  DoD  methods? 
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3.)  Does  Earned  Schedule  provide  more  timely  schedule  predictions  than 
traditional  DoD  methods? 

Scope/Limitations 

The  scope  of  this  research  is  limited  to  Acquisition  Category  I  (ACAT  I) 
programs  from  the  former  Aeronautical  Systems  Center  (ASC)  located  at  Wright- 
Patterson  AFB,  OH.  The  proximity  of  this  product  center  allowed  for  interaction  with  the 
specific  system  program  offices  (SPO). 

One  limitation  to  this  research  is  that  only  ACAT  I  programs  from  a  single 
product  center  are  studied.  Another  potential  limitation  of  this  research  is  its 
applicability:  the  data  come  from  exclusively  DoD  sources,  rendering  the  possibility  that 
the  results  are  germane  only  to  DoD  projects. 

Preview 

With  the  crippling  fiscal  environment  the  DoD  is  currently  experiencing,  coupled 
with  austere  budget  projections  for  the  foreseeable  future,  acquisition  program  managers 
need  to  vastly  improve  on  delivering  an  on-time  product.  However,  as  decision  makers, 
program  managers  are  only  as  good  as  the  information  they  are  given  by  their  analysts. 
The  overall  objective  of  this  research  is  to  identify  the  inadequacies  of  the  current  metrics 
used  to  provide  schedule  analysis  to  program  managers,  as  well  as  to  establish  the  value 
of  earned  schedule  metrics  as  a  more  reliable  toolset  for  the  decision  maker,  through  a 
comparison  of  64  contracts.  With  their  analysts  able  to  provide  more  reliable  predictions 
of  schedule  overages,  as  well  as  more  accurate  forecasts  of  project  completion  dates. 
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program  managers  will  be  far  more  informed  when  making  decisions  in  a  financially  lean 
DoD  environment. 

Chapter  2  of  this  thesis  provides  a  review  of  relevant  literature  and  previous 
research  in  the  fields  of  earned  value  management  and  earned  schedule.  Chapter  3 
discusses  the  methodology  used  to  conduct  this  research.  Chapter  4  discusses  the  results 
of  this  research.  Chapter  5  wraps  up  the  research  through  discussion  of  the  potential 
impact  the  results  will  have  on  DoD  program  management,  and  offers  ideas  of  further 
research  to  explore  in  the  future. 
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II.  Literature  Review 


Introduction 

This  literature  review  evaluates  and  discusses  the  pertinent  theory  and  previous 
research  conducted  in  the  fields  of  Earned  Value  Management  and  Earned  Schedule.  It 
first  examines  broadly  the  importance  of  program  management  within  the  Department  of 
Defense.  It  then  overviews  the  Earned  Value  Management  System  (EVMS)  by  offering  a 
historical  perspective,  defining  the  key  terms  and  components,  explaining  how  EVMS  is 
used  in  the  DoD  today,  and  outlining  the  inherent  flaws  of  using  EVMS  for  schedule 
management.  Next,  it  introduces  Earned  Schedule  (ES),  explains  the  basic  theory, 
discusses  its  current  use  in  program  offices,  and  outlines  the  suggested  advantages  of 
using  ES  over  EVMS  for  schedule  analysis  using  EVM  data.  The  chapter  concludes  with 
a  discussion  of  previous  research  efforts  related  to  ES  application. 

Importance  of  Program  Management  &  Implementation  in  the  DoD 

In  the  DoD’s  Quadrennial  Defense  Review  Report  published  in  February  2010, 
one  of  the  highlighted  topics  was  “Reforming  How  we  do  Business.”  The  pressing  need 
to  improve  how  we  acquire  defense  systems  was  summarized  by  the  following:  “Another 
pressing  institutional  challenge  facing  the  Department  is  acquisitions — broadly  speaking, 
how  we  acquire  goods  and  services  and  manage  the  taxpayers’  money.  Today,  the 
Department’s  obligation  to  defend  and  advance  America’s  national  interests  by,  in  part, 
exercising  prudent  financial  stewardship  continues  to  be  encumbered  by  a  small  set  of 
expensive  weapons  programs  with  unrealistic  requirements,  cost  and  schedule  overruns. 
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and  unacceptable  performance”  (Department  of  Defense,  2010:  75-76).  Of  special 
interest  to  this  research,  the  report  goes  on  to  say  that,  “to  prepare  the  Department  for  the 
complex  threats  that  will  surely  emerge  in  the  future,  we  need  to  make  our  ‘deliberate’ 
processes  more  agile  and  capable  of  responding  to  urgent  needs.  During  periods  of 
conflict,  in  the  traditional  risk  areas  of  cost,  schedule  and  performance,  ‘schedule’  often 
becomes  the  least  acceptable  risk”  (Department  of  Defense,  2010:  81).  Undoubtedly,  the 
need  for  effective  program  management  for  both  cost  and  schedule  has  never  been  more 
vital.  According  to  a  March  2011  GAO  study,  “half  of  the  DoD’s  major  defense 
acquisition  programs  do  not  meet  cost  performance  goals”  (GAO,  2011:  3).  Further, 
“GAO  continues  to  find  that  newer  programs  are  demonstrating  higher  levels  of 
knowledge  at  key  decision  points,  but  most  are  still  not  fully  adhering  to  a  knowledge- 
based  acquisition  approach,  putting  them  at  higher  risk  for  cost  growth  and  schedule 
delays”  (GAO,  2011:  3).  This  same  study  noted  that  “programs  that  modified  key 
performance  requirements  after  development  start  experienced  higher  levels  of  cost 
growth  and  longer  delays  in  delivering  capabilities”  (GAO,  2011:  14).  Specifically  for 
schedule,  programs  who  changed  key  performance  requirements  experienced  “three  to 
five  times  greater  schedule  delays  compared  to  programs  with  unchanged  requirements” 
(GAO,  2011:  14-15). 

These  studies  demonstrate  the  importance  of  sound  program  management  in 
major  defense  acquisition  programs.  Program  management  is  important  because  the  DoD 
operates  with  American  taxpayer  money:  there  are  no  built-in  incentives  to  trim  costs  for 
profit,  which  puts  the  DoD  at  a  competitive  disadvantage.  Program  management  can 
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help  the  DoD  manage  all  the  cost,  schedule  and  technical  performance  aspects  of  large 
DoD  acquisition  programs  (Langhals  interview).  Cost  overruns  and  schedule  delays  are 
inevitable  in  its  absence.  The  question  then  becomes  what  management  technique(s)  are 
best  suited  for  DoD  acquisition  programs. 

There  are  literally  dozens  of  potential  approaches  to  program  management.  The 
most  common  approaches  are  outlined  below.  In  addition  to  the  brief  summary  of  the 
techniques,  arguments  are  made  for  why  the  DoD  chose  the  technique(s)  they  did,  and 
how  each  approach  adds  value  in  the  DoD  context. 

Cost  Engineering  -  The  American  Association  of  Cost  Engineers  (AACE) 
defines  cost  engineering  as  “that  area  of  engineering  practice  where  engineering 
judgment  and  experience  are  utilized  in  the  application  of  scientific  principles  and 
techniques  to  the  problems  of  cost  estimation,  cost  control  and  profitability”  (Clark  and 
Lorenzoni,  1997:  1).  Projects  that  can  be  managed  through  cost  engineering  “can  cover 
anything  from  a  2-day  engineering  effort  to  resolve  a  minor  technical  problem  to  the 
present  day  ‘super  projects’”  (Clark  and  Lorenzoni,  1997:  2).  Cost  engineering  has 
“application  regardless  of  industry  . . .  and  size”  (Clark  and  Lorenzoni,  1997:  1).  Cost 
Engineering  is  an  extremely  effective  management  technique,  but  is  not  specifically  used 
for  major  defense  acquisition  programs.  Cost  engineering  primarily  focuses  on 
profitability  of  a  project,  but  this  focus  is  not  consistent  with  DoD  priorities. 
Additionally,  cost  engineering  is  an  effective  tool  for  managing  cost  but  doesn’t  address 
schedule  and  technical  performance  needed  for  management  of  DoD  programs. 
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Critical  Path  Method  -  The  Critical  Path  Method  (CPM)  is  a  program 
management  tool  that  was  originally  developed  in  the  1950’s  by  the  United  States  Navy. 
The  CPM  is  “a  mathematically  based  algorithm  for  scheduling  a  set  of  project  activities” 
(Santiago  &  Magallon,  2009).  It  is  most  effective  for  a  project  with  “interdependent 
activities”  and  is  “commonly  used  with  all  forms  of  projects,  including  construction, 
software  development,  research  projects,  product  development,  engineering,  and  plant 
maintenance,  among  others”  (Santiago  &  Magallon,  2009).  To  effectively  utilize  the 
CPM,  it  is  essential  to  provide  a  Work  Breakdown  Structure  (WBS),  the  “list  of  all 
activities  required  to  complete  the  project”  (Santiago  &  Magallon,  2009),  as  well  as  to 
outline  the  amount  of  time  required  to  complete  each  task  of  the  WBS.  CPM  calculates 
“the  longest  path  of  planned  activities  to  the  end  of  the  project”  and  “determines  the 
critical  activities  on  the  longest  path”  (Santiago  &  Magallon,  2009).  The  benefits  to 
using  CPM  is  that  it  provides  a  “clear  picture  of  the  scope  of  a  project  that  can  be  easily 
read  and  understood,  it  provides  a  vehicle  for  evaluating  alternative  strategies  and 
objectives,  [shows]  the  interconnections  among  the  job  and  pinpoints  the  responsibilities 
of  the  various  operating  departments  involved”  (Kelley  &  Walker,  2009:  162).  Current 
DoD  practice  in  schedule  analysis  relies  heavily  on  the  critical  path  method  and  the 
integrated  master  schedule  (IMS).  Any  schedule  analysis  tool  used  for  DoD  major 
acquisition  programs  in  the  future  must  address  the  critical  path. 

Critical  Chain  Project  Management  -  Critical  Chain  Project  Management 
(CCPM)  “emphasizes  focusing  on  the  project  schedule  . . .  [and]  reduces  project  changes 
and  the  major  course  of  project  cost  overruns  by  improving  schedule  performance” 
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(Leach,  1999:  39).  CCPM  specifies  the  critical  chain,  rather  than  the  critical  path,  as  the 
project  constraint .  .  .  [it]  includes  resource  dependencies,  and  does  not  change  during 
project  execution”  (Leach,  1999:  39).  CCPM  “seeks  to  change  project  team  behavior, 
encouraging  reporting  early  completion  of  activities  and  elimination  of  multitasking” 
(Leach,  1999:  39).  The  CCPM  project  planning  and  control  process  “directly  addresses 
uncertainty  and  variation  in  project  activity  duration.  It  helps  eliminate  desirable 
behaviors  fostered  by  using  schedule  dates  and  milestones  within  a  project  plan.  It 
focuses  on  developing  and  managing  project  performance  to  meet  or  exceed  reduced 
activity  times,  thereby  reducing  overall  project  duration”  (Leach,  1999:  39).  While 
CCPM  doesn’t  directly  address  cost  performance,  it  is  an  effective  project  management 
technique  for  schedule  planning.  Accurately  predicting  schedule  overages  in  a  timely 
manner  gives  the  program  manager  the  tools  s/he  needs  to  achieve  long  term  project  cost 
savings.  (Leach,  1999:  44-45). 

Earned  Value  Management  -  Earned  Value  Management  is  “the  process  of 
defining  and  controlling  the  project  so  that  defined  objectives  are  met.  The  controlling 
aspect  includes  scope  control,  schedule  control,  and  budget  control”  (Humphreys,  2011: 
32).  Earned  Value  is  “a  management  technique  that  relates  resource  planning  to 
schedule  and  technical  performance  requirements”  (Rose,  9).  Earned  Value  Management 
is  “an  industry  standard  method  of  measuring  a  project’s  progress  at  any  given  point  of 
time,  forecasting  its  completion  date  and  final  cost,  and  analyzing  variances  in  the 
schedule  and  budget  as  the  project  proceeds.  It  compares  the  planned  amount  of  work 
with  what  has  actually  been  completed,  to  determine  if  the  cost,  schedule  and  work 
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accomplished  are  progressing  in  accordance  with  the  plan”  (Lessard  and  Lessard,  2007: 
45). 

Earned  Value  Management  has  shown  its  merit  as  an  effective  management 
technique,  especially  for  cost  performance.  A  study  of  over  700  DoD  acquisition 
programs  shows  that  EVM,  when  applied  early  in  a  programs  life,  can  be  very  effective 
at  predicting  cost  overruns  over  the  life  of  a  program.  Additional  benefits  attributed  to 
EVM  are  that  “it  imposes  discipline  on  the  resource  planning  process  through  the 
development  of  work  and  planning  packages,  provides  a  disciplined,  standardized 
approach  to  measurement  and  terminology,  ties  cost  and  schedule  performance  to 
technical  accomplishment  of  work,  and  provides  and  objective  analysis  of  performance” 
(Rose,  11).  As  the  primary  tool  for  DoD  program  management,  EVM  is  a  proven 
method  to  accurately  capture  a  snapshot  in  time,  and  delivers  very  understandable 
measurements.  There  is  not  a  lot  of  interpretation  needed,  and  is  applicable  to  any  kind 
of  program  (Langhals  Interview). 

EVM  has  demonstrated  its  utility  as  a  program  management  tool  through  a  long 
history  of  DoD  acquisition  programs.  However,  because  it  is  time-consuming  and  costly 
to  implement,  it  is  only  useful  for  certain  types  of  programs.  Programs  where  EVM  is 
most  useful  are  those  with  defined  deliverables  and  products,  programs  with  longer 
durations,  programs  with  strict  budget  limits  (or  Firm  Fixed  Price  contracts),  and 
programs  with  a  single  contract  encompassing  all  or  most  of  the  effort  (Rose,  15). 
Currently,  EVM  is  a  required  tool  for  all  DoD  programs  that  exceed  $20  million  (Rose, 
15). 
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While  there  are  numerous  benefits  to  using  EVM,  there  are  also  drawbacks  to  the 
technique:  a  predominant  shortcoming  to  EYM  is  that  it  can  easily  be  manipulated.  EYM 
is  good  for  monitoring  a  program,  but  it’s  not  very  useful  for  planning.  There  are 
techniques  that  are  better  for  certain  aspects  of  a  program,  but  they  aren’t  as  easily 
traceable  over  time  as  is  EVM  (Langhals  Interview). 

Weighing  the  benefits  and  drawbacks  of  each  of  the  different  program 
management  techniques,  the  DoD  chose  to  use  Earned  Value  Management  as  its 
preferred  approach  for  managing  major  defense  acquisition  programs.  Given  its 
prominence  in  the  DoD,  a  more  detailed  history  of  EVM,  as  well  as  an  introduction  to 
EVM’s  application  to  DoD  programs  follows. 

Earned  Value  Management  System 

In  1967,  the  DoD  issued  a  directive  that  “imposed  35  Cost/Schedule  Control 
Systems  Criteria  (C/SCSC)  on  all  private  industrial  firms  that  wished  to  participate  in 
future  major  government  systems”  acquisition  programs.  The  effect  of  this  mandate  was 
the  formalization  of  the  “earned  value  concept  of  cost  and  schedule  management” 
(Fleming  and  Koppelman,  1998:  19).  The  United  States  Department  of  Defense  has  been 
the  standard  bearer  for  other  countries  to  implement  earned  value  management 
techniques  into  their  major  government  acquisitions.  Earned  Value  Management  can 
give  the  program  manager  “an  early  warning  signal  that  the  project  is  headed  for  a  cost 
overrun  unless  immediate  steps  are  taken  to  change  the  spending  plan”  (Fleming  and 
Koppelman,  1998:  19-20).  While  most  organizations  feel  they  have  a  cost  management 
system  in  place,  “more  than  99%  of  projects  in  the  world  do  not  employ  the  earned  value 
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management  concept.  Instead,  to  monitor  costs,  they  merely  compare  their  spend  plan  to 
their  actual  costs”  (Fleming  and  Koppelman,  1998:  20).  In  recent  years,  C/SCSC  has 
evolved  into  the  Earned  Value  Management  System  (EVMS).  EVMS  has  reduced  the 
number  of  criteria  from  35  to  32,  but  is  still  a  complex,  and  heavily  regulated  governing 
approach  with  substantial  bureaucracy  and  far  too  many  non  value-added  requirements 
(Fleming  and  Koppelman,  1998:  20). 

Earned  Value  Management  is  “a  technique  that  can  be  applied,  at  least  in  part,  to 
the  management  of  all  capital  projects,  in  any  industry,  while  employing  any  contracting 
approach”  (Fleming  and  Koppelman,  2002:  91).  EVMS  is  a  performance  management 
tool  that  integrates  cost,  schedule  and  technical  performance  (Lipke,  2003:  1),  and  has 
shown  to  be  a  reliable  tool  for  measurement  as  early  as  the  15%  completion  point  of  a 
project  (Fleming  and  Koppelman,  2002:  91).  While  Earned  Value  Management  has  a 
proven  record  as  an  effective  tool  for  estimating  a  program’s  cost  performance,  its  ability 
to  predict  schedule  overruns  has  come  into  question. 

Under  EVMS,  there  are  three  measurements  taken  and  two  major  metrics  tracked 
to  perform  a  schedule  analysis  on  a  project.  The  three  measurements  are  the  Budgeted 
Cost  of  Work  Scheduled  (BCWS),  the  Budgeted  Cost  of  Work  Performed  (BCWP)  and 
the  Actual  Cost  of  Work  Performed  (ACWP).  A  graphical  display  of  the  Earned  Value 
measurements,  along  with  other  valuable  EVM  data,  is  illustrated  in  Figure  1. 
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Figure  1:  EVM  Measurements  (Defense  Acquisition  University,  2012) 

BCWS  is  a  “time-phased  budget  baseline  . . .  that  can  be  viewed  as  the  value  to  be  earned 
as  a  function  of  project  work  accomplishments  up  to  a  given  point  in  time”  (Anbari, 

2003:  13).  In  Figure  1  above,  the  BCWS  is  also  shown  as  the  Performance  Management 
Baseline  (PMB).  The  PMB  is  “a  time-phased  budget  plan  for  accomplishing  work  against 
which  contract  performance  is  measured”  (EVMIG,  2006:  103).  BCWP  is  the  “amount 
budgeted  for  performing  the  work  that  was  accomplished  by  a  given  point  in  time” 
(Anbari,  2003:  13).  The  BCWP  is  the  earned  value  of  the  completed  work  scheduled  by 
the  PMB.  ACWP  is  the  cost  in  dollars  of  all  the  work  that  has  been  performed  on  a 
project  (Anbari,  2003:  13).  There  are  three  other  values  illustrated  in  Figure  1:  the 
Estimate  at  Complete  (EAC),  the  Total  Allocated  Budget  (TAB)  and  the  Budget  at 
Complete  (BAC).  The  EAC  is  defined  as  “the  estimated  total  cost  for  all  authorized 
work.  [It  is]  equal  to  the  sum  of  actual  costs  to  date  (including  all  allocable  indirect 
costs),  plus  the  estimated  costs  to  completion”  (EVMIG,  2006:  102).  The  TAB  is 
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defined  as  “the  sum  of  all  budgets  allocated  to  the  contract.  TAB  consists  of  the  PMB 
and  all  management  reserve.  [It]  reconciles  directly  to  the  contract  budget  base” 
(EVMIG,  2006:  104).  The  BAC  is  described  as  “  the  sum  of  all  performance  budgets 
established  for  the  contract.  BAC  is  a  term  that  may  also  be  applied  to  lower  levels,  such 
as  the  PMB  or  at  the  control  level  account”  (EVMIG,  2006:  100). 

Using  these  measurements,  two  schedule  indicators  are  tracked  to  measure  project 
performance.  Schedule  Variance  ($)  (SV  ($))  is  the  computed  cost  difference  of  BCWP 
-  BCWS,  as  seen  in  Figure  1 .  SV($)  is  “a  measure  of  the  conformance  of  actual  progress 
to  the  schedule”  (Anbari,  2003:  14).  Schedule  Variance  can  be  defined  more  simply  as 
“what  got  done  minus  what  was  planned”  (Handshuh,  2006).  An  SV($)  value  greater 
than  zero  indicates  that  the  program  is  ahead  of  schedule.  Conversely,  an  SV($)  value 
less  than  zero  designates  that  a  program  is  behind  schedule.  In  addition  to  SV($),  the 
Schedule  Performance  Index  (SPI($))  is  a  ratio  that  measures  efficiency  for  how  well  the 
project  progresses  relative  to  planning”  (Anbari,  2003:  15).  The  ratio  is  calculated  by 
dividing  the  BCWP  by  the  BCWS  (BCWP/BCWS),  and  details  to  the  user  how  much 
work  has  been  done,  divided  by  how  much  work  was  planned  (Handshuh  2006).  An  SPI 
of  greater  than  1  indicates  that  the  amount  of  work  performed  is  greater  than  was 
originally  planned:  therefore,  the  program  is  ahead  of  schedule.  Conversely,  an  SPI  of 
less  than  1  indicates  that  the  amount  of  work  performed  is  less  than  the  amount  of  work 
scheduled  to  be  performed,  indicating  that  the  program  is  behind  schedule.  The  Actual 
Cost  of  Work  Performed  (ACWP)  is  not  used  directly  in  schedule  analysis,  but  is  an 
integral  measurement  when  performing  an  assessment  on  a  program’s  cost  performance. 
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Measuring  BCWS  and  BCWP,  and  tracking  the  SV  and  SPI  indicators  is  how  program 
managers  using  EVMS  have  traditionally  performed  schedule  analysis. 

In  the  last  decade,  the  validity  of  these  EVMS  indicators  and  their  ability  to 
predict  schedule  performance  has  been  called  into  question,  most  notably  by  Walt  Lipke. 
Predominantly,  “it  is  known  that  [schedule]  indicators  of  EVM  fail  to  provide  good 
information,  nominally,  over  the  final  third  of  the  project;  they  absolutely  break  down  if 
the  project  is  executing  past  its  planned  completion  date  (Lipke,  2003:  1).  Further,  the 
“root  cause  of  the  peculiarities  associated  with  Earned  Value  and  the  EVM  Schedule 
Variance”  is  that  “Earned  Value  is  algebraically  constrained  by  its  ‘budgeted  costs’ 
calculation  reference”  (Henderson,  2003:  2).  The  reason  for  this  is  that,  as  a  program 
moves  towards  its  end  date,  the  schedule  variance  (BCWP-BCWS)  approaches  zero, 
when  the  amount  of  work  performed  closes  in  on  the  amount  of  work  scheduled.  This 
stands  to  reason,  as  the  amount  of  work  accomplished  towards  the  end  of  a  program 
gradually  gets  closer  to  the  amount  of  work  scheduled  for  completion.  Additionally,  the 
Schedule  Performance  Index  (BCWP/BCWS)  approaches  one,  as  the  amount  of  work 
performed  will  eventually  equal  the  amount  of  work  scheduled  (Lipke,  2003:  2).  Thus, 
the  traditional  EVM  schedule  indicators  fail  to  show  that  a  program  is  behind  schedule, 
even  when  we  know  definitively  that  a  program  is  not  completed  on-time.  Beyond  this, 
the  means  of  measuring  schedule  with  EVMS  is  inherently  flawed:  schedule  indicators 
are  measured  in  terms  of  dollars,  rather  than  in  terms  of  time.  This  is  a  mental  obstacle 
that  is  difficult  for  many  program  managers  to  overcome  (Lipke,  2003:  1). 
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In  order  to  illustrate  the  practical  usage  of  the  EYMS  schedule  analysis  tools,  a 


hypothetical  program  has  been  developed,  with  earned  value  data  for  the  program  shown 
in  Table  1.  This  program,  which  begins  on  January  1st,  is  scheduled  to  take  five  months  to 
complete  (End  of  Month,  May).  The  value  of  the  work  planned,  known  as  the  Budgeted 
Cost  of  Work  Scheduled,  was  $200  per  month.  Data  for  the  Budgeted  Cost  of  Work 
Performed  and  the  Actual  Cost  of  Work  Performed  is  also  included.  The  metrics  derived 
from  this  data,  the  Schedule  Variance  and  the  Schedule  Performance  Index  are  also 
provided  in  the  table.  The  final  result  is  a  program  that  was  scheduled  to  only  take  five 
months,  in  fact  took  seven  months  to  complete. 


Table  2.  Earned  Value  Data  for  a  Typical  Program 


January 

February 

March 

April 

May 

June 

July 

BCWS  ($) 

200 

400 

600 

800 

1000 

1000 

1000 

BCWP  ($) 

192 

367 

561 

747 

902 

977 

1000 

ACWP  ($) 

203 

391 

599 

804 

986 

1045 

1112 

SV($) 

-8 

-33 

-39 

-53 

-98 

-23 

0 

SPI($) 

.96 

.92 

.94 

.93 

.90 

.98 

1 

Some  of  the  problems  described  by  Lipke  are  evident  in  this  example.  First,  we  know 
the  program  is  scheduled  to  be  completed  in  five  months,  but  in  fact  took  seven  months. 
However,  as  the  program  gets  closer  to  completion,  the  SV  and  SPI  metrics  become  far 
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less  valuable  when  conducting  a  traditional  EYMS  schedule  analysis.  As  was  previously 
discussed,  a  SV  of  zero  is  notionally  indicative  of  a  program  that  is  on  schedule.  In  our 
example  above,  the  SV  in  June  is  -$23,  and  regresses  to  $0  by  the  end  of  month,  July. 

This  would  indicate  improvement,  but  we  know  that  the  program  is  two  months  over 
schedule.  Second,  it  was  discussed  that  an  SPI  of  one  is  indicative  of  a  program  that  is 
on  schedule.  In  our  example,  between  May  and  July  the  SPI  starts  increasing  as  it 
approaches  1 .  This  leads  the  program  manager  to  believe  that  schedule  performance  is 
actually  improving.  Again,  this  assumption  is  misleading,  because  not  only  is  the 
program  already  over  schedule,  only  $75  worth  of  scheduled  work  was  completed  during 
June.  Because  no  further  work  was  scheduled  during  June,  the  SPI  rose  to  .98.  While 
this  is  a  small  example  with  limited  EVM  data,  it  demonstrates  the  well-known 
inadequacies  of  traditional  schedule  analysis  techniques. 

There  is,  within  Earned  Value  Management,  an  indicator  that  has  been  primarily 
applied  to  cost  metrics,  but  has  begun  to  gain  traction  as  a  useful  tool  for  schedule 
analysis  as  well.  Under  EVM,  the  Cost  Performance  Index  (CPI),  is  “the  ratio  of  the 
earned  value  accrued  divided  by  the  actual  cost”  (Lipke,  2009:  18).  The  new  “companion 
cost  indicator”  (Lipke,  2009:  18)  is  the  To  Complete  Performance  Index  (TCPI),  defined 
as  “the  work  remaining  to  be  accomplished  divided  by  the  amount  of  unspent  funding” 
(Lipke,  2009:  18).  Effectively,  the  TCPI  tells  us  the  “cost  performance  efficiency 
required  for  the  remainder  of  the  project  to  achieve  the  desired  final  cost”  (Lipke,  2009: 
18).  When  the  TCPI  is  “equal  to  or  less  than  1.00,  there  is  confidence”  that  the  project 
can  meet  its  final  cost  (Lipke,  2009:  18).  Further,  if  the  TCPI  is  greater  than  1.10,  the 
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project  is  considered  to  be  “out  of  control”  and  meeting  cost  goals  is  likely  unachievable 
(Lipke,  2009:  18-19).  The  literature  focuses  most  on  TCPI  for  cost,  but  it  also  addresses 
the  TSPI  for  schedule  analysis.  TSPI  is  “equal  to  the  planned  duration  for  the  work 
remaining  divided  by  the  duration  available”  (Lipke,  2009:  21).  All  applications  and 
metrics  for  TCPI  “can  be  made  analogously  to  TSPI”  (Lipke,  2009:  21).  Thus,  a  TSPI 
less  than  or  equal  to  1.00  is  on  track  to  meet  the  project’s  schedule  objectives,  while  a 
program  with  a  TSPI  of  greater  than  1.10  is  very  unlikely  to  be  delivered  on  time. 

Earned  Schedule 

Earned  Schedule  is  a  concept  similar  to  Earned  Value,  but  it  measures  schedule 
performance  in  terms  of  time,  rather  than  in  terms  of  cost.  Earned  Schedule  is 
“determined  by  comparing  the  cumulative  BCWP  earned  to  the  performance  baseline, 
BCWS”  (Lipke,  2003:  5).  Earned  Schedule  was  introduced  in  2003  by  Walt  Lipke,  in  his 
seminal  paper  titled  “Schedule  is  Different.”  Since  then,  analysis  has  been  done  in 
several  areas,  to  include  software  programs  (Lipke,  2008),  construction  projects 
(Rujirayanyong,  2009),  and  small  acquisition  projects  (Lipke,  2011). 

The  first  metric  computed  is  Earned  Schedule,  which  is  “determined  by 
comparing  the  cumulative  BCWP  earned  to  the  performance  baseline,  BCWS.  .  .The 
cumulative  value  of  ES  is  found  by  using  BCWP  to  identify  which  time  increment  of 
BCWS  the  cost  value  occurs”  (Lipke,  2003:  5).  The  value  of  ES  is  “equal  to  the 
cumulative  time  to  the  beginning  of  that  increment,  plus  a  fraction  of  it.  The  fractional 
amount  is  equal  to  the  portion  of  the  BCWP  extending  into  the  incomplete  increment 
divided  by  the  total  BCWS  planned  for  that  same  time  period”  (Lipke,  2003:  5).  As  an 
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example  from  Lipke’s  2003  paper  “Schedule  is  Different,”  the  June  ES  metric  for  a 
project  that  began  in  January  is: 

Equation  1:  Earned  Schedule  Equation  Example 

Earned  Schedule  =  Cumulative  to  May  +  Portion  of  June 

ES  =  5  +  (BCWP($)  -  BCWS(May)/BCWS(June)  -  BCWS(May) 


With  the  ES  value  determined,  other  metrics  of  interest  to  the  program  manager  can  be 
calculated.  The  first  of  these  two  additional  metrics  is  the  Schedule  Variance  in  time 
(SV(t)),  which  is  simply  the  ES  value  minus  by  the  actual  amount  of  time  spent  on  the 
project: 

Equation  2:  SV(t)  Equation 

57(t)  =  ES  -  AT 


The  second  metric  is  the  Schedule  Performance  Index  in  terms  of  time  (SPI(t)),  a  ratio 
defined  as  the  ES  value  divided  by  the  actual  amount  of  time  spent  on  the  project: 

Equation  3:  SPI(t)  Equation 

SPI(t)  =  ES/AT 
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There  is  significant  information  gained  from  these  metrics  that  can  be  of  value  to  the 
program  manager  in  comparison  to  their  EY  counterparts.  First,  as  previously  discussed, 
the  problem  with  SV($)  is  that  it  naturally  regresses  to  zero  as  the  program  gets  closer  to 
completion.  With  SV(t),  the  variance  does  not  regress  to  any  value  and  is  therefore 
useful  throughout  the  life  of  the  program.  Additionally,  unlike  the  traditional  EVM 
SPI($),  the  SPI(t)  metric  does  not  regress  to  one,  offering  further  insight  as  the  program 
approaches  its  conclusion.  This  new  metric  gives  the  program  manager  a  far  more  useful 
schedule  performance  index.  This  new  SPI  is  useful  in  making  predictions  of  estimated 
program  completion  dates. 

The  measurements  taken  when  performing  a  traditional  earned  value  schedule 
analysis  (BCWS,  BCWP,  and  ACWP)  are  also  used  to  calculate  Earned  Schedule 
metrics.  However,  the  metrics  are  computed  in  a  different  way,  in  an  effort  to  put  these 
metrics  in  terms  of  time,  rather  than  in  terms  of  dollars.  The  new  metrics  calculated  are 
Earned  Schedule,  Schedule  Variance  in  terms  of  time,  and  Schedule  Performance  Index 
in  terms  of  time.  The  graphic  below  illustrates  this  difference: 
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Figure  2:  Schedule  Variance  (time)  vs.  Schedule  Variance  (Dollar)  (Lipke,  2006) 

To  interpret  this  graph,  the  starting  place  is  indicated  as  “Time  Now”  on  the  x-axis. 
Under  EVMS,  the  SV($)  is  calculated  by  going  vertically,  and  subtracting  the  earned 
value  from  the  planned  value  (BCWP-BCWS).  Looking  at  the  Y-axis,  this  difference 
gives  us  a  dollar  value.  In  contrast  to  EVMS,  Earned  Schedule  initiates  at  the  actual 
time.  To  calculate  the  SV(t),  the  horizontal  difference  on  the  X-axis  between  the  Earned 
Schedule  and  “Time  Now”  values  capture  the  deviation  in  time.  Regardless  of  whether 
the  program  in  Figure  2  is  analyzed  in  monetary  terms  or  in  terms  of  time,  the  results 
indicate  that  it  is  behind  schedule.  The  earned  value  is  less  than  the  planned  value  for 
SV($),  indicating  the  program  is  behind  schedule  in  monetary  terms.  For  ES,  the  earned 
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schedule  is  behind  the  actual  time  for  the  value  of  SV(t),  indicating  that  the  program  is 
behind  schedule  in  terms  of  time. 

There  are  significant  differences  in  both  the  theory  and  application  of  earned 
value  management  and  earned  schedule,  and  with  that  difference  comes  two  very  singular 
sets  of  terminology.  While  there  are  some  similarities  across  the  two  techniques,  the 
differences  are  clearly  outlined  in  Table  3  below  (Lipke  &  Henderson,  2006:  6). 


Table  3:  Nomenclature  for  EYM  and  ES  (Lipke  &  Henderson,  2003) 


Earned  Value  Management 

Earned  Schedule 

Status 

Earned  Value  (EV) 

Earned  Schedule  (ES) 

Actual  Costs  (AC) 

Actual  Time  (AT) 

Schedule  Variance  (SV) 

SV(t) 

Schedule  Performance  Index  (SPI) 

SPI(t) 

Future  Work 

Budgeted  Cost  for  Work  Remaining 
(BCWR) 

Planned  Duration  for  Work  Remaining 
(PDWR) 

Estimate  to  Complete  (ETC) 

Estimate  to  Complete  (time)  ETC(t) 

Prediction 

Variance  at  Complete  (VAC) 

Variance  at  Completion  (time)  VAC(t) 

Estimate  at  Completion  (EAC)  (supplier) 

Estimate  at  Completion  (time)  EAC(t) 
(supplier) 

Independent  EAC  (IEAC)  (customer) 

Independent  EAC  (time)  IEAC(t) 
(customer) 

To  Complete  Performance  Index  (TCPI) 

To  Complete  Schedule  Performance  Index 
(TSPI) 

One  of  the  main  drawbacks  to  Earned  Schedule  noted  both  in  academia  and  in  the 
program  offices  is  its  inability  to  address  the  critical  path  of  a  project.  In  a  2006  paper. 
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Walt  Lipke  tackles  this  concern  and  dispels  the  notion  that  ES  isn’t  capable  of  addressing 
the  critical  path:  “The  Earned  Schedule  analysis  method  is  demonstrated  ...  to  be 
applicable  to  more  than  the  total  project.  Segregating  and  grouping  EVM  data  for  a 
specific  portion  of  the  project  is  the  technique  by  which  ES  is  made  applicable  to  the  total 
project  and  any  sub-level  desired.  Specifically,  the  technique  is  shown  to  be  capable  of 
analyzing  the  schedule  performance  for  the  critical  path.  By  employing  the  same 
techniques  to  analyze  critical  path,  schedule  performance  by  individual  tasks  can  be 
evaluated,  which  then  allows  identification  of  the  longest  duration  path  for  the  project 
(actual  critical  path)  along  with  schedule  float”  (Lipke,  2006:  7).  Kym  Henderson,  the 
Vice  President  of  Research  and  Standards  with  the  College  of  Performance  Management, 
hypothesized  as  to  why  ES  has  been  thought  to  inadequately  address  the  critical  path: 

“We  have  long  argued  that  ‘drill  down’  on  the  ES  data  is  required  as  part  of  a  detailed 
analysis.  The  issue  from  a  practice  perspective  ...  is  that  the  mainstream  EVM  tool 
vendors  have  not  yet  incorporated  ES  into  their  products  to  do  this,  which  forces 
many/most  practitioners  to  use  add-on  toolsets.”  (Henderson  2012).  Earned  Schedule 
application  to  the  critical  path  will  be  applied  to  an  Acquisition  Category  I  Defense 
Acquisition  Program  in  chapter  four. 

Earned  Schedule  in  Practice 

To  gain  an  understanding  of  how  Earned  Schedule  is  perceived  by  DoD 
practitioners  in  the  field,  and  to  determine  the  extent  of  its  use,  interviews  were  scheduled 
with  the  earned  value  points  of  contact  and/or  the  lead  cost  analyst  on  all  ACAT I 
programs  at  Aeronautical  Systems  Center,  located  at  Wright  Patterson  AFB,  OH. 
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Interviewees  reported  a  wide  range  of  Earned  Schedule  usage  in  the  program  offices: 
some  program  offices  had  never  heard  of  Earned  Schedule,  while  others  used  it 
frequently  in  their  schedule  analysis.  The  first  question  posed  to  the  SPO  was  an  inquiry 
as  to  how  schedule  analysis  was  presently  accomplished.  Generally  speaking,  each  office 
has  an  Earned  Value  “guru”  who  accomplishes  a  full  analysis  on  a  monthly  basis.  This 
analysis  is  conducted  with  the  earned  value  metrics,  and  is  presented  to  the  cost  chief 
and/or  the  program  manager.  When  asked  about  how  earned  schedule  is  used  in  their 
program  office,  the  responses  ranged  from,  “What  is  earned  schedule?”  to  “I’ve  heard  of 
it,  and  I  like  to  use  it  as  a  cross-check.”  A  more  detailed  presentation  of  how  Earned 
Schedule  is  presently  used  in  MDAP  program  offices  appears  in  Chapter  4  of  this 
research. 

While  quantitative  data  was  not  utilized  from  Electronic  Systems  Center  or  Space 
and  Missiles  Center,  qualitative  data  was  made  available.  EVM  analysts  at  each  SPO 
were  asked  what  they  perceived  as  the  value  of  ES,  how  widely  it  was  used  in  their  SPO, 
and  what  they  felt  ES  failed  to  deliver  that  kept  them  from  using  ES.  Of  the  five  analysts 
who  responded,  each  said  they  used  ES  in  some  capacity:  some  used  it  as  a  primary  form 
of  schedule  analysis,  while  others  used  it  primarily  as  an  “indicator.”  ESC  analysts  who 
use  ES  in  their  schedule  analysis  stated  that  the  primary  reason  they  used  it  was  because 
it’s  trustworthiness  in  identifying  the  actual  duration  of  a  project,  as  well  as  defining  the 
schedule  in  terms  of  time.  While  the  EVM  analysts  saw  the  value  in  using  ES,  they 
admitted  that  it  has  not  been  widely  adopted  by  other  personnel  in  the  SPO.  One  analyst 
noted  that  ES’s  predictive  ability  for  large  DoD  programs  has  not  been  studied,  which 
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should  be  alleviated  by  this  research.  Most  analysts  noted  that  ES  is  not  used  because  the 
program  management  community  is  unfamiliar  with  it  and  it  is  not  being  actively 
included  in  the  DAU  curriculum. 

To  demonstrate  the  practical  value  of  Earned  Schedule  compared  to  EVMS,  the 
earlier  example  is  revisited.  When  looking  at  Table  3  below,  there  are  several 
observations  about  the  usefulness  of  ES.  The  first  observation  concerns  the  new  schedule 
variance  calculation.  When  using  EVM,  the  schedule  variance  metric  from  table  1 
showed  that,  at  the  end  of  May,  the  project  was  $-98,  or  $98  behind  schedule.  This 
characterization  of  schedule  variance  in  dollars  is  of  little  value  to  the  program  manager. 
In  contrast,  the  ES  schedule  variance  data  from  Table  2  shows  the  program  manager  that 
the  project  has  an  SV  of  -.49,  or  is  .49  months  behind  schedule.  The  characterization  of 
SV  in  time  is  very  useful  to  the  program  manager.  To  further  illustrate  the  use  of  these 
metrics,  it  is  important  to  look  at  the  program’s  completion  at  the  end  of  July.  Table  1 
shows  us  that  the  schedule  variance  has  regressed  back  to  zero,  leading  the  program 
manager  to  believe  that  the  project  was  delivered  on  time.  It  is  clear,  however,  from  the 
SV  data  in  Table  3  and  the  information  previously  given,  that  the  project  wasn’t 
completed  until  the  end  of  month  July,  when  it  was  scheduled  for  completion  at  the  end 
of  month  May.  This  illustrates  the  two  month  delay  in  delivery,  which  wasn’t  shown  in 
the  EVM  schedule  variance  calculations. 

The  SPI  calculations  for  EVM  and  ES  also  paint  two  very  different  pictures.  An 
SPI  of  greater  than  1  indicates  that  the  program  is  ahead  of  schedule,  while  an  SPI  of  less 
than  1  shows  that  the  program  is  behind  schedule.  For  the  first  five  months  of  the 
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program  (the  estimated  time  until  completion  at  the  start),  the  SPI’s  when  calculated 


using  EVM  were  identical  to  the  calculations  with  ES.  However,  a  deeper  look  shows 
that  at  the  end  of  July  the  SPI($)  for  using  EYM  has  regressed  back  to  1,  leading  the 
program  manager  to  believe  that  the  project  has  been  delivered  on  time.  Again,  from  our 
information,  we  know  that  the  program  was  delivered  two  months  late,  once  again 
rendering  the  EVM  schedule  calculations  misleading.  The  earned  schedule  SPI(t)  shows 
a  value  of  .9  at  the  projected  completion  date,  indicating  that  the  program  is  behind 
schedule. 


Table  4:  Earned  Schedule  Data  for  a  Typical  Program 


January 

February 

March 

April 

May 

June 

July 

BCWS  ($) 

200 

400 

600 

800 

1000 

1000 

1000 

BCWP  ($) 

192 

367 

561 

747 

902 

977 

1000 

ACWP  ($) 

203 

391 

599 

804 

986 

1045 

1112 

ES 

.96 

1.835 

2.805 

3.735 

4.51 

4.885 

7.00 

SV(t) 

-.04 

-.125 

-.03 

-.07 

-.225 

-.625 

1.115 

SPI(t) 

.96 

.9175 

.935 

.934 

.902 

.8142 

1.00 

Previous  Research 

Since  its  inception  in  2003,  a  number  of  studies  have  been  conducted  on  a  variety 
of  different  portfolios  to  establish  the  validity  of  Earned  Schedule.  The  studies, 
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conducted  by  Henderson  (2003),  Rujirayanyong  (2009),  Lipke  (201 1),  Tzaveas, 
Katsavounis  &  Kalfakakou  (2010),  and  Vanhoucke  &  Vandevoorde  (2007)  have  focused 
predominantly  on  portfolios  of  small  programs  and  construction  projects.  No  such 
conclusive  studies  have  studied  the  validity  of  ES  when  applied  to  DoD  AC  AT  I 
programs.  In  2005,  Earned  Schedule  was  studied  by  an  AFIT  student,  1st  Lt  Scott  Smith. 
While  Smith’s  conclusions  did  not  support  the  use  of  ES  as  the  proper  methodology  for 
schedule  analysis,  it  was  because  “the  DAES  historical  database  does  not  contain  the 
necessary  data  to  test  a  hypothesis  under  the  context  proposed  by  Lipke”  (Smith,  2005: 
43).  This  research  builds  upon  Lt  Smith’s  findings  with  the  assistance  of  an  enhanced 
data  set. 

Kym  Henderson  -  “ Earned  Schedule:  A  Breakthrough  Extension  to  Earned 

Value  Theory?  A  Retrospective  Analysis  of  Real  Project  Data ”  (2003) 

After  Walt  Lipke,  in  his  seminal  article  “Schedule  is  Different,”  identified  the 
shortcomings  of  Earned  Value  Management’s  schedule  performance  analysis,  and 
proposed  the  concept  of  Earned  Schedule,  Kym  Henderson  of  the  College  of 
Performance  Management  “retrospectively  applied  the  ES  measures  proposed  by  Lipke 
to  [a]  small  portfolio  of  six  projects  and  subprojects  managed  using  a  ‘simplified’  EVM 
approach”  (Henderson,  2003:  1).  Henderson  concluded  that  “ES  based  schedule  metrics 
more  accurately  portray  a  project’s  schedule  performance  compared  to  the  EVM 
equivalents.  It  is  concluded  that  the  ES  measures  and  metrics  are  expected  to  have  utility 
similar  to  their  cost  based  counterparts,  the  recognized  strength  of  EVM,  and  a  greater 
utility  than  their  historic  EVM  based  equivalents”  (Henderson,  2003:  1).  An  important 
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conclusion  from  Henderson’s  study  confirms  Lipke’s  contention  that  “the  application  of 
Earned  Schedule  provides  a  set  of  schedule  indicators,  which  behave  correctly  over  the 
entire  period  of  project  performance”  (Henderson,  2003:  10).  This  thesis  intends  to  fulfill 
Henderson’s  recommendation  that  “these  conclusions  should  be  validated  with  a  large 
scale  follow  on  study  in  which  ES  is  retrospectively  applied  to  a  broad  portfolio” 
(Henderson,  2003:  1 1)  to  Acquisition  Category  I  (AC AT  I)  Major  Defense  Acquisition 
Programs  (MDAP).  An  ACAT  I  program  is,  according  to  DoD  Instruction  5000.02,  a 
program  with  a  “dollar  value  estimated  by  the  Under  Secretary  of  Defense  for 
Acquisition,  Technology  and  Logistics  (USD(AT&L))  to  require  an  eventual  total 
expenditure  for  research,  development,  test  and  evaluation  (RDT&E)  of  more  than  $365 
million  in  fiscal  year  (FY)  2000  constant  dollars  or,  for  procurement,  of  more  than  $2,190 
billion  in  FY  2000  constant  dollars”  (USD(AT&L),  2008). 

Thammasak  Rujirayanyong  —  “A  comparison  of  Three  Completion  Date 

Predicting  Methods  for  Construction  Projects  ”  (2009) 

In  this  paper,  Rujirayanyong  “compares  three  different  methods  to  predict 
completion  date  of  construction  project  during  construction  stage”  (Rujirayanyong,  2009: 
305).  The  three  methods  compared  are:  the  proposed  method  based  on  neutral  networks, 
earned  schedule,  and  earned  value  management.  Neutral  Network  (NN)  has  “been 
widely  used  as  a  tool  in  different  aspects  of  construction  such  as  estimating,  planning, 
and  productivity  forecasting  . . .  [and]  developed  a  model  [that  considers]  project  physical 
conditions”  (Rujirayanyong,  2009:  307).  Rujirayanyong’s  research  validates  Lipke’s 
assertion  that  “near  the  end  of  the  project,  when  schedule  performance  is  often  the 
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primary  concern,  the  usefulness  of  traditional  schedule  metrics  is  demonstrably  poor” 
(Rujirayanyong,  2009:  312).  The  results  of  this  study  showed  the  NN  model  as  “more 
accurate  and  very  stable  for  project  completion  date  forecasting”  (Rujirayanyong,  2009: 
317).  However,  because  NN  is  not  a  viable  program  management  technique  for  major 
defense  acquisition  programs,  Rujirayanyong’ s  other  results  are  far  more  relevant.  He 
notes  that  “the  ES  method  presented  better  results  than  those  obtained  from  EYM  method 
...  It  is  also  valuable  in  the  last  third  of  the  project,  when  schedule  performance  is  often 
a  crucial  concern  .  .  .  The  EYM  method,  however,  is  still  very  helpful  in  managing 
construction  projects,  particularly  in  cost  control”  (Rujirayanyong,  2009:  317). 
Construction  projects  involve  many  different  inputs  coming  together  to  form  an  output, 
no  different  than  the  many  parts  of  an  aircraft  (avionics,  propulsion,  etc.)  being 
assembled.  The  relevance  of  Earned  Schedule  for  large  construction  projects,  as  shown 
in  Rujirayanyong’ s  study,  lends  credibility  to  the  use  of  ES  for  major  defense  acquisition 
programs. 

Walt  Lipke-  “ Earned  Schedule  Application  to  Small  Projects ”  (2011) 

Lipke  notes  that  the  ES  methods  have  proven  to  perform  very  well.  However, 
“there  are  conditions  during  execution,  generally  for  small,  short  duration  projects,  that 
can  cause  error  in  the  calculated  values  for  the  ES  indicators  and  duration  forecasts  . . . 
The  conditions  are  “down  time,”  defined  as  periods  within  the  schedule  where  no  work  is 
scheduled,  and  “stop  work,”  periods  during  execution  where  management  has  halted 
performance”  (Lipke,  2011:  2).  While  these  conditions  can  adversely  affect  the  ES 
measurements,  “the  ES  calculations  converge  to  the  correct  duration  forecast  and  the 
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final  schedule  variance  result”  (Lipke,  2011:  2).  One  of  the  significant  advantages  of 
applying  ES  is  that  “the  method  provides  the  capability  to  forecast  project  duration  and 
the  expected  completion  date”  (Lipke,  2011:  5).  Lipke’s  paper  concludes  that,  “For  large 
projects,  stop  work  and  downtime  conditions  occurring  for  small  portions  of  the  project, 
in  most  instances,  would  not  have  much  impact  on  the  ES  time  based  indicators  or  the 
duration  and  completion  date  forecasts.  However,  it  is  a  different  matter  for  small 
projects.  The  interrupting  conditions  will  usually  distort  the  ES  indicators  and  forecasts, 
possibly  enough  to  affect  management  decisions”  (Lipke,  2011:  10). 

Tzaveas,  Katsavounis,  Kalfakakou  -  Analysis  of  Project  Performance  of  a 
Real  Case  Study  and  Assessment  of  Earned  Value  and  Earned  Schedule 
Techniques  for  the  Prediction  of  Project  Completion  Date”  (2010) 

A  680km  highway  stretching  from  one  side  of  Greece  to  the  other  is  one  of  14 
priority  projects  of  the  ELI,  and  the  first  to  be  designed  and  built  to  modem  international 
specifications  (Tzaveas,  Katsavounis  &  Kalfakakou,  2010:  754).  For  this  study,  a  bridge 
carries  the  road  over  a  deep  river  valley,  and  the  schedule  of  the  project  was  broken  into  9 
control  periods  that  took  into  account  the  percentage  completion  of  work  and  main  events 
of  the  project  (Tzaveas,  Katsavounis  &  Kalfakakou,  2010:  755-756).  The  results  support 
Lipke  and  Henderson’s  main  takeaways:  “It  is  apparent  that  Earned  Schedule  converges 
faster  towards  the  actual  duration  without  any  remarkable  results  . . .  The  tendency  of 
SPI($)  to  reach  a  value  of  1.0  is  clearly  noted  even  though  the  project  is  late  and 
exceeded  the  original  contractual  deadline,  and  that  occurs  at  about  66%  completion,  that 
is  when  2/3  of  the  project  is  effectively  complete”  (Tzaveas,  Katsavounis  &  Kalfakakou, 
2010:  756).  Further,  the  authors  noted  that  the  “analyses  carried  out  reveal  the 
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superiority  of  the  ES  method  showing  reliable  results  during  the  whole  project  duration  .  . 
.  [and]  provides  value  and  reliable  results  along  the  project’s  lifespan”  (Tzaveas, 
Katsavounis  &  Kalfakakou,  2010:  758).  This  study  effectively  illustrated  not  only  the 
applicability  of  ES  to  large  construction  projects,  but  also  the  acceptance  of  earned 
schedule  principles  on  an  international  scale. 

Mario  Vanhoucke  and  Stephan  Vandevoorde  -  “ Measuring  the  Accuracy  of 
Earned  Value/Earned  Schedule  Forecasting  Predictors”  (2007) 

Because  of  the  limited  real-life  data  set,  Vanhoucke  and  Vandevoorde  aimed  to 
validate  “various  earned  value  based  forecasting  methods  on  a  very  large  and  diverse  set 
of  fictive  projects”  (Vanhoucke  &  Vandevoorde,  2007:  27).  They  simulated  the 
performance  of  three  forecasting  methods  “by  simulating  a  large  dataset  containing 
projects  of  moderate  size  and  calculating  the  forecast  accuracy  of  each  method” 
(Vanhoucke  &  Vandevoorde,  2007:  27).  The  authors  tested  the  Anbari’s  Planned  Value 
method,  Jacob’s  earned  duration  method,  and  Lipke’s  earned  schedule  method.  The 
results  of  their  simulation  “clearly  reveals  that  the  earned  schedule  method  outperforms, 
on  average,  the  two  other  forecasting  methods  . . .  both  for  projects  that  finish  ahead  of 
schedule  and  projects  with  a  delay”  (Vanhoucke  &  Vandevoorde,  2007:  28).  A  major 
drawback  to  the  earned  value  method  is  the  inaccuracy  of  SPI($)  as  an  indicator  over  the 
final  third  of  a  project:  Vanhoucke  and  Vandevoorde ’s  research  confirmed  this,  while 
also  noting  the  “late  stage  forecast  accuracy  is  much  better  for  the  ES  method  compared 
to  the  PV  and  ED  methods  . . .  The  SPI(t)  indicator  of  the  earned  schedule  method  is 
developed  to  overcome  [the  EV  method’s]  quirky  behavior,  leading  to  an  improved 
forecast  accuracy  at  the  end  of  the  project”  (Vanhoucke  &  Vandevoorde,  2007:  28). 
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This  study  successfully  simulated  the  effectiveness  of  ES  compared  to  other  forecasting 
methods  for  a  large  portfolio  of  fictitious  programs. 

The  above  mentioned  papers  have  extensively  studied  the  application  of  Earned 
Schedule  in  a  variety  of  different  program  sizes  and  applications.  However,  a  study  of 
Earned  Schedule’s  value  to  major  defense  acquisition  programs  within  the  Department  of 
Defense  has  never  been  adequately  conducted.  This  research  intends  to  fill  that  void  in 
the  literature. 

Summary 

This  chapter  identified  the  importance  of  program  management,  introduced 
popular  management  techniques,  established  how  EVM  is  used  within  the  DoD  and 
discussed  why  it  may  not  be  the  best  method  for  schedule  analysis.  Further,  this  chapter 
introduced  the  theory  of  Earned  Schedule,  detailed  the  work  pioneered  by  Lipke  and 
Henderson,  and  introduced  a  new  set  of  metrics  that  may  improve  the  DoD’s  ability  to 
manage  schedule  delays  using  EVM  data.  The  next  chapter  seeks  to  take  the  literature 
reviewed  here,  introduce  the  data  collected,  and  identify  the  methods  used  to  analyze  the 
data. 
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III.  Methodology 


Introduction 

The  purpose  of  this  chapter  is  to  outline  the  methodology  used  for  the  research.  It 
will  discuss  the  data  source,  the  limitations  of  the  data,  hypothesis  of  the  research,  and  the 
statistical  process  used  to  perform  the  analysis. 

Data  Source 

The  Office  of  the  Under  Secretary  of  Defense  (OUSD)  for  Acquisition, 
Technology,  and  Logistics  (AT&L)  maintains  the  Defense  Acquisition  Management 
Information  Retrieval  (DAMIR)  system.  DAMIR  is  comprised  of  all  contractor 
performance  report  (CPR)  data  for  all  Department  of  Defense  (DoD)  acquisition 
programs.  The  CPR  “is  a  contractually  required  report,  prepared  by  the  contractor, 
containing  performance  information  derived  from  the  internal  EVMS.  [It]  provides  status 
of  progress  on  the  contract”  (EVMIG  2006,  91).  Within  this  CPR  data  are  monthly  and 
quarterly  updates  for  all  the  Work  Breakdown  Structures  (WBS)  for  a  project.  The  CPR 
is  broken  down  into  “five  formats  containing  data  for  measuring  contractors’  cost  and 
schedule  performance  on  Department  of  Defense  acquisition  contracts”  (OUSD(AT&L), 
2005:  1).  Detailed  explanations  of  each  of  the  five  formats  follows:  “Format  1  provides 
data  to  measure  cost  and  schedule  performance  by  product-oriented  Work  Breakdown 
Structure  elements,  the  hardware,  software,  and  services  the  Government  is  buying. 
Format  2  provides  the  same  data  by  the  contractor’s  organization  (functional  or 
Integrated  Product  Team  (IPT)  structure).  Format  3  provides  the  budget  baseline  plan 
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against  which  performance  is  measured.  Format  4  provides  staffing  forecasts  for 
correlation  with  the  budget  plan  and  cost  estimates.  Format  5  is  a  narrative  report  used  to 
explain  significant  cost  and  schedule  variances  and  other  identified  contract  problems  and 
topics”  (OUSD(AT&L),  2005:  1).  The  CPR  “is  a  management  report  [that]  provides 
timely,  reliable  summary-level  data  with  which  to  assess  current  and  projected  contract 
performance.  The  CPR’s  primary  value  to  the  Government  is  its  ability  to  reflect  current 
contract  status  and  reasonably  project  future  program  performance”  (OUSD(AT&L), 
2005:  1).  See  appendix  A  for  examples  of  Format’s  1-5  of  a  contractor  performance 
report. 

For  this  research,  several  measurements  will  be  gathered  from  the  Format  1  of 
each  month’s  CPR.  All  data  was  cumulative  for  the  program: 

Budgeted  Cost  of  Work  Scheduled  (BCWS)  ->  Planned  Value 
Budgeted  Cost  of  Work  Performed  (BCWP)  ->  Earned  Value 
Schedule  Variance  in  terms  of  Dollars  (SV($))  ->  Earned  Value  -  Planned  Value 

Schedule  Performance  Index  in  terms  of  Dollars  (SPI($))  ->  Earned 
Value/Planned  Value 

These  figures  are  provided  to  the  Systems  Program  Office  (SPO)  by  the  contractor  every 
month,  and  are  validated  by  SPO  cost  analysis  personnel.  The  Defense  Contract 
Management  Agency  (DMCA)  is  the  “DoD  Executive  Agent  for  EVMS.  [They]  . . . 
ensure  initial  and  ongoing  compliance  with  EVMS  guidelines  in  ANSI/EIA-748” 
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(EVMIG,  2006:  18).  Such  guidelines  ensure  the  EVM  data  produced  by  the  contractors 
allows  program  managers  to  make  the  most  informed  decision. 

Data  was  collected  from  two  sources.  The  first  source  is  the  Cost  Library  located 
at  Aeronautical  Systems  Center  (ASC),  Wright-Patterson  AFB,  OH.  The  CPR  data  from 
the  cost  library  was  collected  on  microfiche.  While  it  would  have  been  ideal  to  use  the 
pure  CPR  data  from  the  program  offices  as  the  sole  data  source,  it  was  determined  over 
the  course  of  the  research  that  the  data  was  incomplete:  entire  years  of  data  were  missing 
for  many  programs,  and  for  some  programs  only  data  from  a  single  contract  existed.  The 
data  that  was  available  from  the  program  offices  was  monthly  data,  but  the  data  as  a 
whole  was  far  less  complete  than  the  data  gathered  from  DAMIR.  There  were  some 
programs,  however,  that  had  usable  data  from  the  program  office:  of  particular  note,  the 
A- 10.  The  second  source  of  data  was  DAMIR.  The  programs  selected  were:  Rockwell 
B-1B  Lancer,  Northrop  Grumman  B-2  Spirit,  General  Dynamics  F-16  Fighting  Falcon, 
McDonnell  Douglas  F-15  Eagle,  Fairchild  Republic  A- 10  Thunderbolt,  Fairchild  T-46, 
Boeing  E-3  Sentry  Airborne  Warning  and  Control  System  (AW ACS),  Boeing  C-17 
Globemaster  III,  General  Dynamics/Grumman  EF-111A  Raven,  AGM-131  Short  Range 
Attack  Missile  (SRAM)  II,  AGM-86  Air  Launch  Cruise  Missile  (ALCM),  AGM-65 
Maverick,  and  Lockheed  Martin  C-130J  Super  Hercules  Upgrade.  Additionally,  DAMIR 
was  a  focal  point  for  lists  of  active  and  completed  Air  Force  programs,  provided  program 
start  dates,  original  Program  Manager  Estimated  Completion  Dates  (PMECD),  and  final 
completion  dates. 
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The  programs  comprising  the  dataset  have  been  completed,  and  are  either  in  their 
operational  phase,  or  have  been  retired  from  the  Air  Force  fleet.  The  preceding  programs 
are  all  Acquisition  Category  I  (ACAT  1)  Major  Defense  Acquisition  Programs  (MDAP): 
An  ACAT  I  program  is,  according  to  DoD  Instruction  5000.02,  a  program  with  a  “dollar 
value  estimated  by  the  Under  Secretary  of  Defense  for  Acquisition,  Technology  and 
Logistics  (OUSD(AT&L))  to  require  an  eventual  total  expenditure  for  research, 
development,  test  and  evaluation  (RDT&E)  of  more  than  $365  million  in  fiscal  year  (FY) 
2000  constant  dollars  or,  for  procurement,  of  more  than  $2,190  billion  in  FY  2000 
constant  dollars”  (OUSD(AT&L),  2008). 


In  addition  to  the  above  mentioned  programs,  a  “deep-dive”  analysis  will  be 
completed  on  the  C-130J.  While  the  twelve  programs  mentioned  above  will  be  analyzed 
at  the  “program”  level,  the  C-130J  will  be  analyzed  at  the  work  breakdown  structure 
(WBS)  level.  Table  5  illustrates  the  complete  dataset. 


Table  5:  Platforms,  Number  of  Contracts  &  Number  of  Data  Points  in  Data  Set 


Platform  Name 

Number  of  Contracts 

Number  of  Data  Points 

B-l 

14 

233 

B-2 

2 

19 

F-16 

4 

51 

F-15 

8 

115 

A-10 

5 

135 

E-3 

2 

50 

T-46 

2 

25 

C-17 

11 

226 

EF-111 

2 

36 

AGM-131  (SRAM) 

1 

17 

AGM-86  (ALCM) 

8 

74 

AGM-65  (Maverick) 

4 

41 

C-130J 

1 

65 

Sum 

64 

1,087 

39 


Data  &  Limitations 


There  are  some  limitations  to  this  data,  predominantly  in  the  breadth  of  the  study. 
The  study  is  limited  to  only  projects  with  complete  CPR  data,  as  it  is  necessary  to  have 
complete  and  accurate  figures  for  the  budgeted  cost  of  work  scheduled  (BCWS)  and 
budgeted  cost  of  work  performed  (BCWP).  For  this  reason,  no  active  acquisition 
programs  can  be  studied,  and  no  programs  with  incomplete  and/or  inaccurate  data  can  be 
studied.  A  further  limitation  to  some  of  the  programs  selected  is  that  only  “Cost  Plus,” 
and  other  similar  types  of  contracts  can  be  studied.  Cost  Plus  contracts  encapsulate  the 
costs  of  the  materials  and  labor,  as  well  as  an  additional  award  fee  or  incentive  fees. 
Several  contracts,  and  even  some  entire  programs,  are  Firm  Fixed  Price  (FFP)  contracts. 
In  these  contracts,  the  government  and  the  contractor  agree  on  a  set  of  goods  and  services 
that  need  to  be  provided  by  the  contractor  for  a  flat  fee  paid  by  the  government.  For  FFP 
contracts,  EVM  data  is  not  collected:  a  firm  fixed  price  contract  outlines  specific  services 
to  be  accomplished  for  a  pre-determined  amount  of  money,  negating  the  need  for  any 
earned  value  analysis. 

Air  Force  acquisition  is  predominantly  accomplished  at  three  major  product 
centers,  each  under  the  Air  Force  Life  Cycle  Management  Center.  A  product  center 
focuses  on  acquiring  systems  for  a  particular  Air  Force  capability.  For  instance, 
Aeronautical  Systems  Center,  the  largest  of  the  three  product  centers,  “designs,  develops 
and  delivers  dominant  aerospace  weapon  systems  and  capabilities  for  the  United  States 
Air  Force,  other  United  States  military,  allied  and  coalition-partner  war  fighters,  in 
support  of  Air  Force  leadership  priorities”  (WPAFB  ASC  factsheet).  A  limitation  to  this 
study  is  that  the  data  comes  only  from  one  service,  the  United  States  Air  Force,  and  from 
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a  single  product  center,  ASC,  at  Wright-Patterson  AFB,  OH.  This  limits  the  results  of 
this  finding  to  aircraft  acquisition,  and  it  is  possible  that  the  results  may  not  be 
extrapolated  to  other  types  of  weapons  systems. 

Within  the  dataset  of  aeronautical  systems,  there  were  additional  limitations. 
First,  no  data  was  available  for  rotary- wing  aircraft  (helicopters)  in  either  the  cost  library 
or  the  DAMIR  system.  Additionally,  no  data  was  found  for  other  programs,  specifically 
the  Lockheed  Martin  F-22  Raptor,  the  McDonnell  Douglass  KC-10  Extender,  and  the 
Boeing  KC-135  Stratotanker. 

Hypothesis 

The  hypotheses  for  this  research  explore  two  distinctive  avenues  of  program 
performance:  timeliness  and  accuracy.  The  first  facet  attempts  to  answer  whether  using 
Earned  Schedule  for  a  program’s  schedule  analysis  would  identify  schedule  overages 
with  greater  accuracy  than  the  current  Earned  Value  Management  schedule  analysis 
techniques.  The  second  facet  attempts  to  answer  whether  using  Earned  Schedule  for  a 
program’s  schedule  analysis  would  identify  schedule  overages  earlier  in  the  life  cycle  of 
a  program  than  the  current  Earned  Value  Management  schedule  analysis  techniques. 

H0:  Earned  Schedule  is  not  a  more  accurate  and  timely  predictor  of  schedule  overages 
than  traditional  Earned  Value  techniques. 

Ha:  Earned  Schedule  is  a  more  accurate  and  timely  predictor  of  schedule  overages  than 
traditional  Earned  Value  techniques. 
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In  failing  to  reject  the  null,  the  results  of  this  analysis  would  indicate  that  using  Earned 
Schedule  rather  than  EYM  for  schedule  management  offers  no  additional  benefit.  If, 
however,  the  null  is  rejected,  Earned  Schedule  proves  to  be  a  more  accurate  and  timely 
predictor  of  schedule  overages  than  traditional  EVM  methods. 

Assumptions 

There  was  one  major  assumption  made  during  this  analysis,  regarding  what 
constitutes  a  program  in  “trouble.”  There  is  no  specific  SPI  value  agreed  upon  by  earned 
value  and  earned  schedule  practitioners  that  signals  that  major  changes  must  be  made  to  a 
program  in  order  to  meet  schedule  milestones.  There  is,  however,  an  agreed  upon  value 
for  the  To  Complete  Schedule  Performance  Index  (TSPI).  Practitioners  agree,  based  on 
Lipke’s  (2009)  analysis,  that  any  TSPI  greater  than  1.1  signals  a  program  must  undergo 
significant  changes  in  order  to  meet  the  scheduled  completion  date.  In  essence,  a  TSPI 
value  of  1.1  tells  us  that  10%  more  work  must  be  completed  each  month  for  the  programs 
duration  to  have  an  SPI  of  1  at  the  programs  completion  date.  Because  there  is  not  a 
universally  accepted  SPI  value,  the  10%  “behind”  schedule  logic  was  used.  For  this 
research,  any  program  with  an  SPI  less  than  .9  was  considered  to  be  “in  trouble.” 

Statistical  Process 

Earned  Value  and  Earned  Schedule  statistics  are  needed  to  test  the  hypotheses. 
Each  contract’s  quarterly  BCWS  and  BCWP  figures  are  used  to  calculate  the  SV($), 
SV(t),  SPI($)  and  SPI(t)  for  that  respective  month.  While  the  formulas  for  SV($)  and 
SPI($)  have  been  previously  defined,  the  formulas  for  the  earned  schedule  metrics  have 
not.  Two  key  elements  are  needed  for  these  calculations:  the  earned  schedule  value,  and 
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the  actual  time  elapsed  in  the  project.  Earned  schedule  is  “the  cumulative  earned  value  in 
time  units  identified  on  the  BCWS  curve,  as  established  by  the  value  of  the  BCWP 
[where}  partial  units  of  time  are  calculated”  (Haupt,  2004:  5).  ES  accounts  for  the  value 
of  completed  months,  as  well  as  partial  months.  The  ES  statistic  is  calculated  using  the 
formula  below: 


Equation  4:  Earned  Schedule  Equation 


ES  =  Monthx  +  (BCWP cum-  BCWSx') /(BCWSy  -  BCWSx) 

Once  the  ES  statistic  has  been  calculated  it  is  used,  along  with  the  actual  duration 
(abbreviated  as  AT)  of  the  program,  to  determine  the  values  of  SV(t)  and  SPI(t): 

Equation  5:  Earned  Schedule  Metrics 


5V(t)  =  ES  -  AT 

SPl(t)  =  ES/AT 

With  the  data  collected  and  the  Earned  Value  and  Earned  Schedule  statistics 
calculated,  the  originally  postulated  hypothesis  can  be  examined  through  a  series  of  tests 
First,  a  paired  t-test  is  employed  to  determine  whether  there  is  a  statistically  significant 
difference  between  the  EV  and  ES  data.  A  paired  t-test  “is  used  to  ascertain  how  likely 
the  difference  between  two  means  that  contain  the  same  (or  matched)  observations  is  to 
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occur  by  chance  alone”  (Bausell  &  Li,  2002:  57).  The  paired  t-test  is  used  when:  “(1) 
there  are  two  continuous  sets  of  numbers,  and  (2)  the  hypothesis  to  be  tested  is  expressed 
in  terms  of  a  mean  difference  between  these  two  sets  of  numbers”  (Bausell  &  Li,  2002: 
57).  For  this  research,  the  pair  t-test  takes  the  difference  between  the  SPI(t)  and  SPI($) 
and  analyzes  whether  the  difference  between  the  two  is  significantly  different  than  zero. 
To  test  this  significance,  the  SPI(t)  value  and  SPI($)  value  for  each  data  point  is 
compared.  Effectively,  the  SPI($)  value  is  the  pre-test  variable,  and  the  SPI(t)  is  the  post¬ 
test  variable.  Once  the  comparison  is  completed  on  these  variables,  the  t-statistic  is  tested 
for  significance.  If  the  significance  is  determined  to  be  below  an  alpha  level  of  .05 
(Bausell  &  Li,  2002:  10),  then  it  can  be  concluded  that  there  is  not  a  statistically 
significant  difference  between  the  Earned  Value  and  Earned  Schedule  methods.  If  a 
statistical  significance  is  not  found,  then  it  must  be  concluded  that  ES  does  not  provide 
more  timely  or  accurate  schedule  information  than  EV.  If  statistical  significance  is 
found,  then  further  tests  are  conducted  to  determine  whether  ES  provides  more  accurate 
and  timely  schedule  information  than  EV. 

Upon  conclusion  of  the  paired  t-test,  five  additional  tests  are  conducted  on  the 
data  for  each  of  the  contracts.  These  five  tests  use  descriptive  statistics  to  help  answer 
the  original  hypothesis  questions. 

Test  1:  Analysis  of  SPI(t)  and  SPI($)  below  .90  Over  Time 

The  first  test  analyzes  the  SPI(t)  for  Earned  Schedule  and  the  SPI($)  for  Earned  Value, 
and  determines  which  method  detects  that  the  program  is  “in  trouble”  earlier  in  the 
program’s  life.  A  program  was  determined  to  “in  trouble”  when  the  SPI  dropped  below 
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.90.  A  definitive  value  for  SPI  signaling  that  a  program  is  “in  trouble”  was  not  found 
during  the  literature  review  for  this  research,  so  an  assumption  was  made.  To  analyze 
this,  a  scatter  plot  is  utilized  for  each  of  the  individual  contracts,  capturing  the  SPI(t)  and 
SPI($)  values  at  similar  durations  of  the  program’s  life.  This  scatter  plot  captures  the 
activity  of  the  SPI(t)  and  SPI($)  values  over  the  life  of  the  contract. 

Test  2:  Analysis  of  Frequency  of  SPI(t)  and  SPI($)  values  below  .90 

The  second  test  examines  the  percentage  of  time  that  the  SPI(t)  and  SPI($)  values  for  all 
of  the  contracts  are  below  .90.  Effectively,  this  test  analyzes  whether  one  method  more 
consistently  indicates  that  a  program  is  “in  trouble.”  In  addition  to  the  results  of  each 
individual  contract,  a  summary  histogram  is  utilized  for  both  the  Earned  Value  and 
Earned  Schedule  methods.  The  histogram  shows  the  percentage  distribution  of  SPI(t) 
and  SPI($)  values  below  .90,  and  also  provides  the  mean  and  median  values  for  each 
method. 

Test  3:  Analysis  of  Optimism  and  Accuracy  in  SPI(t)  vs.  SPI($)  values 

The  third  test  is  broken  into  two  parts:  both  parts  also  use  the  SPI  values  for  Earned 
Schedule  and  Earned  Value  Management.  The  first  part  of  this  test  looks  at  each  data 
point  and  determines  which  method  is  the  more  “optimistic”  of  the  two,  regardless  of 
whether  the  SPI  value  was  above  or  below  .90.  For  instance,  if  SPI(t)  was  .95  and  SPI($) 
was  .90,  it  was  determined  that  SPI(t)  was  more  optimistic  for  this  data  point. 

Conversely,  if  SPI(t)  was  .82  and  SPI($)  was  .97,  it  was  determined  that  SPI($)  was  more 
optimistic.  The  total  number  of  “optimistic”  data  points  for  each  method  was  calculated 
at  the  end.  Historically,  major  acquisition  programs  have  failed  to  meet  schedule 
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objectives  because  of  optimistic  planning:  this  test  determines  which  method  more 
accurately  identifies  the  more  optimistic  scheduling.  The  second  part  of  the  test  looks  at 
which  method  is  more  accurate.  For  each  data  point,  the  SPI  values  are  compared  to  the 
final  schedule  result,  and  whichever  method  is  determined  to  be  closer  to  the  final 
contract  over/under  run  is  deemed  to  be  the  more  “accurate”  method.  The  total  number 
of  more  accurate  points  for  each  method  is  totaled  at  the  end,  and  displayed  as  a 
percentage. 

Test  4:  Analysis  of  TSPI  Values 

The  fourth  test  looked  at  a  different  metric:  the  To  Complete  Schedule  Performance 
Index  (TSPI).  The  TSPI  is  the  SPI  efficiency  that  must  be  achieved  from  the  current 
month  until  the  estimated  project  completion  date  in  order  to  come  in  on  time.  Any 
program  with  a  TSPI  value  greater  than  1.1,  which  implies  that  10%  more  work  needs  to 
be  completed  every  month  than  is  scheduled,  is  considered  to  be  unrecoverable.  This 
research  investigates  whether  a  program  that  had  a  TSPI  reach  a  value  of  1.1  ever 
recovered  to  come  in  on  schedule.  The  same  test  is  done  for  programs  where  the  TSPI’s 
reached  1.05  and  1.01.  Once  analysis  on  all  programs  is  completed,  a  histogram  is 
utilized  to  show  the  distribution  of  program  completion  percentages.  Further,  the  mean 
and  median  completion  percentage  for  each  TSPI  value  is  calculated.  This  test  aims  to 
determine  which  TSPI  value  the  DoD  should  use  to  indicate  a  program  is  unrecoverable 
unless  changes  are  made. 
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Test  5:  Analysis  of  SV(t)  vs.  SV($)  Divergence  Point 

The  final  test  uses  SV(t)  and  SV($)  data  for  all  of  the  contracts  studied.  As  discussed  in 
chapter  two,  one  of  the  advantages  of  ES  is  that  it  puts  schedule  in  terms  of  time,  rather 
than  dollars.  The  schedule  variance  under  EVM  (SV($))  is  a  dollar  figure,  the  difference 
between  Earned  Value  and  Planned  Value.  At  the  end  of  a  program,  all  work  that  was 
scheduled  will  be  performed.  Thus,  the  SV($)  will  always  regress  to  zero.  However,  ES 
puts  schedule  in  terms  of  time:  the  value  tells  us  how  far  ahead  or  behind,  in  terms  of  our 
unit  of  measure  (in  this  case,  quarters),  our  program  is.  A  negative  SV(t)  indicates  that 
the  program  is  behind  schedule,  while  a  positive  SV(t)  indicates  that  a  program  is  ahead 
of  schedule.  For  example,  an  SV(t)  of  -1.45  indicates  that  the  program  is  1.45  quarters 
behind  schedule.  To  analyze  how  the  two  schedule  variances  react,  a  scatter  plot  with 
two  y-axes  is  utilized.  One  y-axis  is  measured  in  dollars  to  accommodate  the  EV 
calculation.  The  other  y-axis  is  measured  in  time  to  accommodate  the  ES  calculation. 

The  purpose  of  this  test  is  to  graphically  show  how  the  SV’s  for  a  program 
compare  when  measured  using  the  two  different  methodologies.  Because  the  behavior  of 
the  two  schedule  variance  methodologies  had  already  been  identified  in  the  ES  literature, 
this  research  also  results  in  an  analysis  of  at  which  percentage  completion  point  in  a 
contract  the  divergence  between  the  two  methods  occurs.  To  study  this,  the  schedule 
variance  scatter  plot  was  used  to  subjectively  determine  at  which  point  in  the  programs 
completion  the  SV(t)  began  to  decrease  at  an  increasing  rate  while  the  SV($)  began  to 
regress  towards  zero. 
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Chapter  Summary 

The  purpose  of  this  chapter  is  to  outline  the  methodology  used  for  the  research.  It 
discusses  the  data  source,  the  limitations  of  the  data,  hypothesis  of  the  research,  and  the 
statistical  process  used  to  perform  the  analysis.  Chapter  Four  will  discuss  and  describe 
the  results  of  this  analysis. 
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IV.  Analysis  and  Results 


Chapter  Overview 

The  purpose  of  this  chapter  is  to  document  the  results  of  the  research.  We  first 
looked  back  to  the  research  questions  submitted  in  Chapter  1. 

1. )  To  what  extent  is  Earned  Schedule  currently  utilized  in  Air  Force  ACAT  I 

acquisition  programs? 

2. )  Does  Earned  Schedule  provide  more  accurate  schedule  predictions  than 

traditional  DoD  methods? 

3. )  Does  Earned  Schedule  provide  more  timely  schedule  predictions  than 

traditional  DoD  methods? 

The  first  question  looks  to  qualitatively  answer  how  extensively  ES  is  used  in  Air  Force 
acquisition  programs,  as  well  as  to  gain  a  deeper  understanding  of  how  schedule  analysis 
is  conducted  and  of  what  value  it  is  to  the  program  manager.  The  two  final  questions  are 
quantitative  in  nature.  Once  it  is  determined  that  there  is  a  statistical  difference  between 
Earned  Value  Management  and  Earned  Schedule,  a  series  of  tests  is  conducted  to  answer 
the  questions  of  which  method  is  more  accurate  and  more  timely  as  a  predictor  of 
schedule  performance. 

Extent  of  Earned  Schedule  in  Air  Force  Acquisition  Category  I  Programs 

Before  statistically  analyzing  the  difference  between  Earned  Value  Management 
and  Earned  Schedule,  we  investigate  our  first  research  question  to  gain  an  understanding 
of  to  what  extent,  and  how,  schedule  analysis  is  currently  conducted  in  Air  Force  System 
Program  Offices  (SPO),  as  well  as  the  SPO’s  familiarities  with  the  concept  of  Earned 
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Schedule.  Interviews  with  SPO’s  at  Aeronautical  Systems  Center  at  Wright  Patterson 
AFB,  OH  were  conducted  in  person,  as  well  as  through  e-mail  and  phone  conversation  as 
required.  Due  to  funding  limitations,  interviews  with  program  offices  at  Electronic 
Systems  Center  at  Hanscom  AFB,  MA  and  Space  and  Missile  Center  at  Los  Angeles 
AFB,  CA  were  conducted  exclusively  through  e-mail  communication. 

Most  programs,  especially  AC  AT  I  programs,  have  an  Earned  Value  analyst. 
These  Earned  Value  analysts  were  the  focal  point  for  the  interviews  and  data  collection. 

In  the  absence  of  an  EV  analyst,  we  worked  with  the  programs  Finance  chief.  We  first 
explored  how  the  program  conducted  their  schedule  analysis,  and  how  frequently  the 
analysis  was  done.  We  ascertained  where  they  got  their  schedule  data,  what  EVM 
calculations  were  done,  and  how  the  results  were  presented  to  the  program  manager.  We 
then  determined  how  familiar  the  SPO  was  with  the  concepts  of  Earned  Schedule,  and 
whether  ES  was  used  in  their  analysis.  If  the  SPO  did  not  use  ES,  we  asked  what  they 
felt  it  lacked  to  make  it  a  valuable  tool.  We  asked  the  SPO  if  they  were  satisfied  with 
their  results  using  EVM  as  their  schedule  analysis  tool,  and  if  they  felt  there  were  any 
drawbacks  to  its  use.  Finally,  we  asked  the  analysts’  opinion  as  to  why  they  felt  Earned 
Schedule  had  not  gained  popularity  as  a  schedule  analysis  tool  for  defense  acquisition 
programs. 

Earned  Value  Management  in  Practice 

Most  of  the  programs  surveyed  conduct  their  schedule  analysis  on  a  monthly 
basis.  Some  stated  that  they  have  experimented  with  conducting  the  analysis  on  a  weekly 
basis,  but  that  it  offered  no  further  insight  and  thus  was  not  worth  the  additional  effort. 
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Programs,  almost  exclusively,  receive  their  data  from  the  Integrated  Master  Schedule 
(IMS),  which  is  also  the  basis  for  the  Contractor  Performance  Reports  (CPR).  The  SPO’s 
briefed  their  EYM  results  to  the  Program  Manager  on  a  monthly  basis:  most  did  so  in 
face-to-face  meetings,  while  others  provided  the  results  in  an  e-mail  and  discussed  any 
areas  of  concern  on  a  teleconference. 

One  area  of  disparity  between  the  programs  was  how  much  value  they  felt  the 
Program  Manager  placed  in  the  EVM  results.  One  EV  analyst  stated  that  “The  program 
manager  is  proficient  in  understanding  how  the  schedule  status  influences  the  Estimate  at 
Complete  (EAC)  projections,  and  I  think  the  program  manager  understands  the  value  of 
EVM”  (Tatum).  Another  EVM  analyst  agreed  with  this  sentiment,  by  stating  that  “The 
program  manager  sees  the  value  of  Earned  Value  Management.  Maybe  not  as  much  the 
numbers,  but  they  find  the  analysis  to  be  valuable.”  (King).  The  EVM  analyst  for 
another  program  stated  that  “Our  program  manager  was  very  fluent  and  interested  in 
every  detail  of  the  EVM  analysis  . . .  well  beyond  the  standard  reporting”  (Winterhalter). 
One  other  analyst  supported  these  thoughts:  “Our  program  manager  is  very  fluent  in  the 
different  schedule  metrics  and  takes  necessary  action  to  improve  the  IMS  on  the  contract. 
They  also  use  the  lessons  learned  to  ensure  the  IMS  on  the  follow-on  contract  will  be  a 
better  tool  to  manage  their  program.”  (Suchland)  However,  one  analyst  disagreed  with 
these  statements:  they  felt  that  “the  program  manager  is  only  really  interested  in  the 
schedule  when  it  starts  to  overrun.”  (Kapaku).  While  there  were  some  dissenting 
opinions,  EVM  analysts  largely  felt  their  program  managers  understood  their  work  and 
saw  its  value  in  the  scheme  of  their  program. 
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Earned  Schedule  in  Practice 


The  responses  from  each  Product  Center  were  varied.  Generally,  Earned 
Schedule  has  not  been  broadly  accepted  by  government  acquisition  programs.  We  asked 
each  of  the  SPO’s  if  they  were  familiar  with  Earned  Schedule.  We  asked  those  who  were 
familiar  with  ES  to  expound  upon  how  they  use  ES  in  their  analysis  and  how  they  feel  it 
compares  to  the  results  obtained  using  EVM.  The  results  of  the  inquiry  are  displayed  in 
Table  6  below: 


Table  6:  Results  of  Earned  Schedule  Usage  Qualitative  Study 


Product  Center 

Total  Number 

of  Program 

Offices  in 

Product 

Center 

Total  Number  of 

Program  Offices 

Contacted 

Number  of 

Program  Office’s 

Responded 

Number  of 

Program  Offices 

Using  ES 

ASC 

16 

14 

12 

3 

ESC 

30 

30* 

5 

3 

SMC 

12 

1** 

1 

0 

*  -  Includes  ACAT  II  and  ACAT  ] 

[II  programs  located  at  Hanscom  AFB 

**  -  Response  from  SAF/FMC  staff  on  behalf  of  all  programs  within  the  product  center 

*  -  Includes  ACAT  II  and  ACAT  III  programs 

As  Table  6  illustrates,  Earned  Schedule  has  not  broadly  been  accepted  by  Air 
Force  program  offices.  At  Aeronautical  Systems  Center  (ASC)  and  Electronic  Systems 
Center,  Earned  Schedule  has  been  introduced  to,  and  is  understood  by,  several  of  the 
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program  offices,  but  all  were  adamant  that  they  did  not  use  ES  as  their  primary  schedule 
analysis  tool.  Space  and  Missile  Center,  contrarily,  has  not  implemented  ES  in  any 
capacity,  and  it  is  not  currently  used  by  any  of  their  programs. 

For  those  programs  that  replied,  the  responses  ranged  from  “I  have  never  heard  of 
Earned  Schedule,”  to  “I  am  very  familiar  with  Earned  Schedule,  and  I  use  it  extensively 
in  my  analysis.”  Of  the  seventeen  responses,  fourteen  of  the  programs  have  heard  of 
Earned  Schedule.  Of  those  fourteen  programs,  six  of  the  programs  actively  use  Earned 
Schedule  in  their  analysis.  None  of  the  programs  queried  felt  there  were  significant 
limitations  to  using  EVM  exclusively:  rather,  they  all  endorsed  EVM,  and  feel  that  it 
provides  more  than  enough  valuable  information  to  the  program  manager.  The  strongest 
endorsement  for  the  implementation  of  Earned  Schedule  comes  from  the  KC-46  program, 
who  noted  “We  have  implemented  Earned  Schedule  analysis  on  our  program.  We 
include  the  SPI(t)  calculation  on  our  performance  charts.  So  far,  the  ES  numbers  have 
not  been  much  different  than  the  EV  numbers,  but  SPI(t)  becomes  a  more  useful  index 
later  in  the  program.”  (Pierri).  Another  program,  the  C130J,  also  uses  Earned  Schedule 
in  their  analysis:  “We  find  Earned  Schedule  to  be  useful  because  its  puts  into  perspective 
how  far  behind  we  are.  We  use  ES  as  a  crosscheck  for  EVM.”  Further,  the  C-130J  stated 
that  they  find  ES  to  be  “more  useful  at  the  WBS  level  because  it  helps  identify  where 
delays  occur.”  (King).  One  analyst  from  ESC  noted  that  ES  is  “fairly  trustworthy  as  far 
as  identifying  an  actual  duration  for  the  schedule  variance  -  SV(t).  The  SV(t)  usually 
would  be  close  to  what  was  pulled  out  of  the  Integrated  Master  Schedule  (IMS).” 

Another  ESC  analyst  responded,  “ES  adds  another  element  of  understanding  as  a 
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snapshot  of  past  vs.  potential  contract  performance.  It  bridges  the  gap  between  a 
schedule  metric  being  described  in  a  time  component  as  opposed  to  a  monetary 
component.”  However,  that  same  analyst  responded  that  “ES  is  not  used  in  any  capacity 
that  I  am  aware  of  on  my  contracts.”  Of  the  eighteen  programs  who  responded,  only  six 
use  Earned  Schedule  in  their  analysis,  and  none  use  it  as  their  primary  schedule  analysis 
technique. 

Based  on  our  findings,  it  is  clear  that  the  use  of  ES  within  DoD  acquisition 
programs  is  very  scarce.  We  surveyed  the  programs  as  to  why  they  felt  ES  had  not 
gained  any  popularity  since  its  introduction  in  2003.  One  common  response  was 
obtained  when  asked  why  ES  is  not  used  as  the  primary  tool  for  schedule  analysis  in  the 
program  offices:  unfamiliarity.  Several  analysts  stated  that  while  they  felt  that  ES  added 
value,  it  is  not  given  the  same  support  from  program  leadership  as  is  EVMS.  Several 
hypothesized  that  because  it  is  not  included  in  the  curriculum  at  the  Defense  Acquisition 
University  (DAU),  it  is  not  widely  understood,  and  therefore  not  utilized  in  the  field. 

Revisiting  the  original  research  question,  we  have  determined  the  extent  of 
Earned  Schedule  usage  in  Air  Force  ACAT  I  programs.  Rather  than  utilize  Earned 
Schedule,  programs  use  the  IMS  to  complete  their  monthly  EVM  analysis.  In  general, 
program  managers  are  content  with  these  EVM  metrics.  Thus,  the  extent  of  ES  usage  is 
minimal.  The  integration  of  Earned  Schedule  as  the  primary  schedule  analysis  tool  has 
yet  to  happen  for  any  programs,  but  it  is  used  as  a  secondary/crosscheck  tool  for  about 
one-third  of  the  programs  surveyed. 
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Accuracy  &  Timeliness  of  Earned  Schedule  vs.  Earned  Value  Management 

Prior  to  examining  the  accuracy  and  timeliness  of  Earned  Schedule  versus  Earned 
Value  Management,  we  must  first  determine  if  there  is  a  difference  between  the  two 
methods.  If  there  is  a  statistical  difference  between  the  two  methods,  we  reject  the  null 
hypothesis,  and  further  compare  the  two  methods. 

Results  of  Significance  Test 

To  determine  if  the  results  of  the  Earned  Value  Management  and  Earned  Schedule 
techniques  were  statistically  significantly  different  from  zero,  a  paired  t-test  is  used.  The 
paired  t-test  measures  the  mean  difference  between  two  sets  of  numbers.  For  this 
analysis,  a  comparison  of  13  programs,  comprised  of  64  contracts,  resulting  in  1,087  data 
points  was  used.  The  resulting  t-statistic  of  the  paired  t-test  is  -8.6231,  with  a  resulting  p- 
value  of  2.27467E-17  and  a  rejection  of  the  null  hypothesis.  Therefore,  it  is  determined 
that  the  difference  between  Earned  Value  Management  and  Earned  Schedule  is 
statistically  significant.  The  output  of  the  paired  t-test  is  found  in  Figure  3  below. 
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t-Test:  Paired  Two  Sample  for 

Means 

Variable  1 

Variable  2 

Mean 

0.939165476 

0.95750293 

Variance 

0.008831643 

0.006653895 

Observations 

1087 

1087 

Pearson  Correlation 

0.689419981 

Hypothesized  Mean 

Difference 

0 

df 

1086 

tStat 

-8.623145392 

P(T<=t)  one-tail 

1.13734E-17 

t  Critical  one-tail 

1.646257934 

P(T<=t)  two-tail 

2.27467E-17 

t  Critical  two-tail 

1.962150792 

Figure  3:  Results  of  Paired  t-test 


The  result  of  the  significance  test  demonstrates  that  there  is  a  statistical  difference 
between  Earned  Value  Management  and  Earned  Schedule.  With  this  fundamental 
difference  established,  we  revisit  our  original  hypothesis  questions,  and  analyze  whether 
Earned  Schedule  or  Earned  Value  Management  is  more  accurate  and  timely  in  predicting 
schedule  overages.  The  questions  are  examined  through  a  series  of  five  tests. 


Test  1:  Analysis  of  SPI(t)  and  SPI($)  below  .90  Over  Time 

The  first  test  conducted  shows  how  the  SPI(t)  and  SPI($)  acted  over  the  course  of 
the  program.  The  literature  suggests  that  Earned  Value  Management  and  the  SPI($)  is 
relatively  useless  over  the  final  third  of  a  program  because  the  SPI($)  value  regresses 
towards  zero.  For  this  research,  the  earned  value  and  planned  value  data  for  each  point 
were  used  to  calculate  the  SPI(t)  and  SPI($),  and  a  scatter  plot  is  used  to  illustrate  how 
the  SPI(t)  and  SPI($)  changed  over  the  course  of  a  program.  Figure  4  exhibits  the  scatter 
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plot  for  a  typical  program,  in  this  case  the  BIB  Propulsion  Lot  1  contract.  See  Appendix 


B  for  SPI(t)  vs.  SPI($)  scatter  plots  for  all  programs. 


BIB  Propulsion  Lot  1  SPI(t)  vs.  SPI{$) 
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Figure  4:  BIB  Propulsion  Lot  1  SPI(t)  vs.  SPI($) 


While  graphs  such  as  that  seen  in  Figure  4  are  helpful  to  see  how  the  SPI(t)  and 
SPI($)  values  react  over  the  course  of  each  individual  program,  not  all  programs  reacted 
the  same  way.  To  get  a  more  broad  understanding  of  how  the  SPI(t)  and  SPI($)  values 
for  defense  acquisition  programs  reacted  over  time,  the  SPI(t)  and  SPI($)  values  for  each 
program  were  calculator  at  the  20%,  40%,  50%,  60%,  80%,  and  90%  program 
completion  points.  From  there,  the  average  for  each  completion  point  was  graphed  to 
show  the  reaction  of  the  broader  portfolio.  That  graph  is  seen  in  Figure  5. 
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Figure  5:  SPI  Comparison  Over  Time 


The  SPI  comparison  in  Figure  5  shows,  at  the  portfolio  level,  what  we  expected 
from  the  SPI  values.  The  SPI(t)  and  SPI($)  values  react  with  general  consistency  over 
the  first  two-thirds  of  the  program,  with  the  SPI  values  separated  by  only  hundredths  of  a 
point.  However,  at  the  two-thirds  completion  point,  the  SPI  values  go  in  different 
directions:  the  SPI(t)  value  begins  to  decrease  at  a  more  rapid  pace,  while  the  SPI($) 
value  regresses  back  towards  a  value  of  1.0.  We  know  that  an  SPI  value  of  1.0  indicates 
that  a  program  is  on  schedule,  but  also  that  all  of  the  programs  studied  were  delivered 
after  their  estimated  completion  dates.  This  validated  our  belief  that  as  a  program 
progresses  past  the  two-thirds  completion  point,  SPI($)  becomes  an  unreliable  measure  of 
program  schedule.  While  the  first  test  shows  how  the  SPI(t)  and  SPI($)  values  react 
over  the  life  of  a  program  at  both  the  individual  and  enterprise  level,  we  wanted  to 
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determine  which  method  of  schedule  analysis  was  more  accurate  in  detecting  schedule 


overages. 

Test  2:  Analysis  of  Frequency  of  SPI(t)  and  SPI($)  values  below  .90 

The  second  test  utilizes  the  graphs  from  test  1  to  determine  the  point  at  which  the 
initial  indication  came  that  there  were  issues  with  a  program.  The  intent  of  this  test  is  to 
determine  whether  Earned  Value  Management  or  Earned  Schedule  is  an  earlier  detector 
of  problems  in  meeting  program  schedule  objectives.  To  perform  this  analysis,  the  point 
at  which  the  SPI(t)  and  SPI($)  for  each  program  first  dropped  below  .90  is  determined. 
Because  the  literature  lacked  a  conclusive  threshold,  the  definition  of  a  problem  (SPI(t)  or 
SPI($)  <  .90)  is  an  interpretation  for  this  research.  That  information  is  presented  in  Table 
7  below: 


Table  7:  Comparison  of  First  Data  Points  Below  .90 


Percent  Complete 
(Mean) 

Programs  with  no  SPI 
value  below  .90 

Earned  Schedule 

29.89% 

21 

Earned  Value 
Management 

17.89% 

30 

From  the  results,  we  determined  that,  on  average,  Earned  Value  Management 
detected  earlier  in  a  program  that  there  are  problems.  This  was  a  notable  result,  but  one 
that  lost  some  significance  when  it  was  found  that  EVM  failed  to  detect  any  problems  30 
of  the  64  programs  studied,  while  Earned  Schedule  only  failed  to  detect  a  problem  on  21 
of  the  programs.  This  told  us  that  there  were  nine  programs  where  Earned  Schedule 
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detected  a  problem  where  EVM  never  did.  We  also  noticed  that  there  were  several 
programs  where  the  SPI(t)  and/or  SPI($)  value  dipped  below  .90  early  in  a  program 
(perhaps  the  first  data  point),  but  quickly  recovered  to  an  SPI  value  of  greater  than  .90  for 
the  remainder  of  the  program.  An  example  of  this  is  shown  in  Figure  6  below: 


A-10  Production  Option  1 


♦  SPI(t) 
■  SP1($) 


Figure  6:  Example  of  Quickly  Recovering  SPI  Value 


We  felt  this  first  test  failed  to  paint  the  full  picture,  necessitating  a  second  test: 
determining  which  method  more  frequently  indicates  that  there  are  problems  with  a 
program. 

The  second  part  of  this  test  examines  the  amount  of  time  that  a  program  was 
deemed  to  be  “in  trouble”  to  determine  if  Earned  Schedule  or  Earned  Value  Management 
is  more  consistent  at  identifying  schedule  delays.  To  study  this,  the  SPI(t)  and  SPI($) 
values  for  each  data  point  are  calculated.  From  there,  the  total  number  of  points  with  an 
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SPI  value  below  .90  is  counted  for  each  method.  Of  the  1,087  data  points,  12.42%  (135 
points)  had  an  SPI($)  of  less  than  .90  under  the  Earned  Value  Management  method,  while 
20.24%  (220  points)  had  an  SPI(t)  of  less  than  .90  under  the  Earned  Schedule  method. 

The  results  are  shown  in  Table  8. 


Table  8:  Total  Number  of  Data  Points  with  SPI  Less  than  .90 


Total  Points  <  .90 

Percentage  of  Overall 
Points 

Earned  Value 
Management 

135 

12.42 

Earned  Schedule 

220 

20.24 

The  total  number  of  SPI(t)  and  SPI($)  values  below  .90  were  gathered  at  each  of 
the  following  program  completion  points:  20%,  40%,  50%,  60%,  80%,  and  90%.  The 
results  are  found  in  Table  9  below. 


Table  9:  Number  of  SPI  Values  Below  .90  Over  Time 


20% 

40% 

50% 

60% 

80% 

90% 

SPI(t) 

20 

17 

11 

14 

15 

20 

SPI($) 

12 

11 

4 

5 

2 

1 

These  values  were  calculated  to  show  the  differing  reactions  of  the  SPI  values 
over  time.  While  Earned  Schedule  is  marginally  more  accurate  in  predicting  schedule 
delays  over  the  first  half  of  a  program,  there  was  a  considerable  change  over  the  second 
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half  of  the  programs  that  was  consistent  with  the  literature.  As  a  program  got  closer  to  its 
completion  point,  the  SPI($)  metric  indicated  fewer  programs  were  having  problems.  At 
the  60%  completion  point,  SPI(t)  found  14  programs  to  have  SPI  values  below  .90  while 
SPI($)  found  only  5  programs  to  have  problems.  At  the  80%  completion  point,  SPI(t) 
found  15  programs  to  have  problems  while  SPI($)  only  found  2.  Finally,  at  the  90% 
completion  point,  SPI(t)  found  20  programs  to  be  “in  trouble,”  while  SPI($)  found  only  1 . 
This  confirmed  what  was  found  in  the  literature  and  what  we  found  in  earlier  tests:  as  a 
program  approaches  its  completion  point,  Earned  Value  Management  yields  an  SPI($) 
value  that  approaches  1.0,  indicating  that  the  program  is  on  schedule.  We  also  noticed 
that  there  were  20  points  detected  at  both  the  20%  and  90%  completion  points  for  SPI(t). 
Deeper  analysis  revealed  that  seven  data  points  which  detected  a  problem  at  the  20% 
completion  point  also  detected  one  at  the  90%  completion  point.  This  test  revealed  that 
as  a  program  nears  its  completion  point,  the  use  of  Earned  Schedule  is  a  much  more 
accurate  schedule  predictor  than  the  current  method  employed  by  the  Department  of 
Defense. 

The  data  from  Table  9  is  displayed  graphically  in  Figure  7.  Figure  7  compares  the 
total  number  of  SPI(t)  and  SPI($)  values  below  .90  at  six  thresholds  over  time.  It 
demonstrates  how,  especially  over  the  final  half  of  the  program,  the  SPI(t)  is  a  much 
more  accurate  predictor  of  schedule  overages.  At  the  90%  completion  point,  only  one 
program  is  determined  to  be  behind  schedule  using  EVMS,  while  ES  predicts  schedule 
overages  for  20  programs. 
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Figure  7:  Comparison  of  Numbers  of  SPI  Values  Below  .90 


The  raw  data  suggests  that  Earned  Value  Management  is  an  earlier  indicator  of 
schedule  overages  than  Earned  Schedule,  as  it  detects  problems  at  the  17.89%  program 
completion  point  as  opposed  to  the  29.89%  completion  point.  However,  this  singular 
statistic  is  misleading:  overall,  EVM  fails  to  detect  a  problem  on  nine  fewer  programs 
than  ES,  and  detects  85  fewer  SPI  values  below  .90  than  ES.  Ultimately,  Earned 
Schedule  is  the  better  metric  for  schedule  detection  over  the  life  of  an  acquisition 
program. 


Test  3a:  Analysis  of  Optimism  and  Accuracy  in  SPI(t)  vs.  SPI($)  values 

Historically,  schedule  metrics  are  often  overly  optimistic.  Programs,  especially 
those  of  the  magnitude  of  an  ACAT  I  defense  acquisition  program,  are  typically  delivered 
later  than  the  original  estimate,  but  the  numbers  calculated  on  a  monthly  basis  paint  a 
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much  more  positive  picture.  We  conducted  this  test  to  determine  which  method  was  the 


more  “optimistic  of  the  two,  regardless  of  whether  the  SPI  value  was  above  or  below  .90. 
Whichever  SPI  value  is  higher  is  determined  to  be  the  most  optimistic.  Of  the  1,087  data 
points,  the  SPI($)  value  for  Earned  Value  Management  was  more  optimistic  59.43%  (646 
points)  of  the  time,  while  the  SPI(t)  value  for  Earned  Schedule  was  more  optimistic 
35.05%  (381  points)  of  the  time.  There  are  60  instances  (5.52%)  where  the  SPI($)  and 
SPI(t)  values  are  the  same.  Earned  Value  Management  was  41.02%  ((646-38 1)/646) 
more  optimistic  than  Earned  Schedule.  The  results  of  this  test  are  reflected  in  Table  10. 


Table  10:  Points  more  Optimistic 


Number  of 
Occurrences 

Percentage  of 
Overall 

Earned  Value 
Management 

646 

59.43 

Earned  Schedule 

381 

35.05 

EVM  =  ES 

60 

5.52 

As  a  way  to  show  graphically  the  difference  between  the  two  methods,  two 
histograms  were  used.  The  first,  shown  in  Figure  8,  depicts  when  EVM  was  the  more 
optimistic  of  the  two  methods.  On  the  X-axis  is  the  percentage  of  points  that  were  more 
optimistic  in  a  program,  with  the  frequency  (in  the  number  of  programs)  on  the  y-axis. 
Figure  8  illustrates,  for  example,  that  there  are  14  programs  where  Earned  Value 
Management  is  the  more  optimistic  measure  100%  of  the  time.  Further,  Figure  8 
demonstrates  that  56  of  the  64  total  programs  studied  had  over  50%  of  their  data  points 
where  the  SPI($)  value  was  the  more  optimistic  of  the  two  measures. 
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EVM  More  Optimistic 
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■  Frequency 


Figure  8:  EVM  More  Optimistic 

Figure  9  illustrate  the  percentage  of  points  in  a  program  where  Earned  Schedule 
was  the  more  optimistic  measure.  There  is  a  substantial  skew  to  the  left,  indicating  that 
most  of  the  programs  studied  had  less  than  50%  of  their  data  points  where  Earned 
Schedule  was  the  more  optimistic  measure.  There  were  only  two  programs  out  of  the  64 
studied  where  Earned  Schedule  was  the  more  optimistic  measure  on  more  than  75%  of 
the  data  points. 


65 


ES  More  Optimistic 
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Figure  9:  ES  More  Optimistic 


One  further  illustration  for  this  test  is  displayed  in  Figure  10,  which  compares  the 
frequency  with  which  each  SPI  value  is  more  optimistic  for  each  program.  While  there 
are  some  programs  where  the  SPI(t)  value  is  more  frequently  the  optimistic  measure,  the 
overwhelming  number  of  programs  had  more  SPI($)  values  that  were  more  optimistic. 
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Overall  Frequency  of  Optimistic  Data  Points  by  Method 


Program  Number 


Figure  10:  Overall  Frequency  of  Optimistic  Data  Points  by  Method 


A  final  demonstration  for  this  test  is  illustrated  in  Figure  1 1 .  This  graph  shows 
which  method  was  more  optimistic  as  a  schedule  predictor  at  various  points  throughout 
the  life  of  a  program:  20%,  40%,  50%,  60%,  80%,  and  90%.  Because  of  the  number  of 
months  in  certain  programs,  there  wasn’t  always  an  exact  90%  completion  point,  so  the 
closest  possible  point  was  used  for  that  program.  It  should  be  noted  that  some 
completion  points  have  greater  than  64  points,  the  number  of  programs  studied  in  this 
research.  For  instance,  the  20%  completion  point  has  72  points  because  8  programs  had 
idential  SPI(t)  and  SPI($)  values  at  that  point.  The  results  of  this  test  revealed  that  there 
isn’t  much  difference  in  the  first  50%  of  a  program:  the  SPI($)  value  is  only  slightly  more 
optimistic  than  the  SPI(t)  value.  However,  there  comes  a  staggering  change  over  the 
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second  half  of  a  program.  At  the  60%  completion  point,  EVM  was  more  optimistic  for 
46  programs,  compared  to  ES  which  was  more  optimistic  for  only  22  programs.  At  the 
80%  completion  point,  the  difference  in  these  two  values  grows:  SPI($)  is  more 
optimistic  for  53  programs,  whereas  SPI(t)  is  more  optimistic  for  only  1 1  programs.  At 
the  90%  completion  point.  Earned  Value  Management  was  more  optimistic  for  59  of  the 
programs,  while  Earned  Schedule  was  more  optimistic  for  only  5  programs.  We  surmise 
that,  had  we  had  the  final  CPR  reports  as  a  part  of  our  data  set,  that  this  trend  would  have 
continued  and  EVM  would  have  been  more  optimistic  for  close  to,  if  not  all,  of  the 
programs  studied  in  this  research. 


Comparison  of  More  Optimistic  Method  Over 

Time 


Figure  11:  Comparison  of  More  Optimistic  Method  Over  Time 


Upon  examining  the  results  of  this  test,  we  concluded  that  overall,  the  SPI($) 
metric  and  Earned  Value  Management  system  were  41.02%  more  optimistic  as  a 
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measure  of  a  program’s  schedule.  While  the  difference  between  the  two  methods  early 
in  the  program  is  not  as  noteworthy,  there  is  significant  difference  in  the  two  values  over 
the  final  half  of  a  program.  This  test  helped  us  answer  our  second  research  question,  and 
leads  to  the  conclusion  that  Earned  Schedule  is  a  more  accurate  schedule  predictor  for  Air 
Force  ACAT  I  defense  acquisition  programs. 

Test  3b:  Analysis  of  SPI(t)  vs.  SPI($)  Points  Closer  to  the  Final  Schedule  Result 

The  previous  test  was  one  technique  to  research  whether  Earned  Value 
Management  or  Earned  Schedule  was  a  more  accurate  schedule  predictor.  As  an 
additional  means  of  measuring  the  accuracy  of  the  EVM  and  ES  methods,  the  SPI($)  and 
SPI(t)  values  for  each  data  point  are  compared  to  the  final  schedule  result.  Whichever 
method  is  closer  to  the  final  contract  over/under  run  is  deemed  to  be  the  more  “accurate” 
technique.  Of  the  1,087  data  points  studied,  Earned  Value  Management  was  closer  to  the 
final  schedule  result  37.07%  (403  points)  of  the  time,  while  Earned  Schedule  was  the 
more  accurate  technique  57.41%  (624  points)  of  the  time.  There  are  60  instances 
(5.52%)  where  the  SPI($)  and  SPI(t)  values  are  the  same.  A  summary  of  the  data  points 
closer  to  the  final  schedule  result  are  outlined  in  Table  1 1  below: 
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Table  11:  Points  Closer  to  Final  Schedule  Result 


Number  of 
Occurrences 

Percentage  of 
Overall 

Occurrences  (%) 

Earned  Value 
Management 

403 

37.07 

Earned  Schedule 

624 

57.41 

EVM  =  ES 

60 

5.52 

The  graphic  in  Figure  12  depicts  the  frequency  of  programs  having  a  particular 
percentage  of  their  data  points  closer  to  the  final  schedule  result.  For  instance,  the  C130J 
Weapon  System  program  had  14  points  where  the  SPI($)  was  closer  to  the  final  schedule 
result  than  the  SPI(t).  There  were  62  data  points  for  this  program,  so  EYM  was  closer  to 
the  final  schedule  result  22.58%  of  the  time.  According  to  Figure  12,  this  program  was 
one  of  nine  programs  where  the  SPI($)  value  was  closest  to  the  final  schedule  result 
between  20%  and  25%  of  the  time.  There  is  a  definite  skew  to  the  right  in  this  histogram, 
with  48  of  the  64  programs  having  less  than  50%  of  their  data  points  closer  to  the  final 
schedule  result. 
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EVM  Closer  to  Final 


%  of  Points  Closer  to  Final  Result  in  a  Program 


■  Frequency 


Figure  12:  EVM  Closer  to  Final 


While  Figure  12  shows  the  frequency  of  SPI($)  values  being  closer  to  the  final 
schedule  result,  Figure  13  shows  the  same  information  for  the  SPI(t)  values.  To  use  a 
different  example,  the  SPI(t)  value  was  closer  to  the  final  schedule  result  for  32  of  the  46 
(69.57%)  data  points  for  the  C-17  Research,  Development,  Test  &  Evaluation  and  Lot  1 
Production  program.  According  to  Figure  13,  this  program  was  one  of  seven  programs  to 
have  between  65%  and  70%  of  its  data  points  with  an  SPI(t)  value  closer  to  the  final 
schedule  result.  As  this  graphic  mirrors  the  graphic  in  Figure  12  (above),  there  is  a  skew 
to  the  left.  There  are  only  four  programs  that  have  less  than  30%  of  their  data  points  with 
SPI(t)  values  closer  to  the  final  schedule  result. 
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ES  Closer  to  Final 


%  of  Points  Closer  to  Final  Result  in  a  Program 


■  Frequency 


Figure  13:  ES  Closer  to  Final 


In  addition  to  analyzing  the  programs  at  an  individual  level,  we  also  wanted  to 
determine  how  the  entire  portfolio  acted  over  a  period  of  time.  As  evidenced  in  Figure 
14,  there  was  little  difference  over  the  first  40%  of  a  program,  the  SPI(t)  was  closer  to  the 
final  schedule  result  over  the  final  60%  of  the  program.  A  small  dip  occurred  at  the  80% 
completion  point,  where  SPI(t)  was  closer  to  the  final  schedule  for  only  nine  more 
programs  than  SPI($),  after  having  been  closer  on  fourteen  programs  at  the  60% 
completion  mark.  This  trend  quickly  reversed,  as  the  SPI(t)  value  was  closer  to  the  final 
schedule  result  for  seventeen  more  programs  (39  vs.  22)  than  the  SPI($)  value  at  the  90% 
completion  point. 
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Figure  14:  Comparison  of  SPI  Closer  to  Final  Over  Time 

In  summary,  the  tests  comparing  the  optimism  in  ES  vs.  EVM  and  which  method 
is  closer  to  the  final  schedule  result  help  answer  the  original  research  questions  regarding 
accuracy  of  the  Earned  Schedule  and  Earned  Value  Management  methods.  We  previously 
determined  that  Earned  Schedule  more  frequently  determined  problems  with  a  program 
than  Earned  Value  Management,  and  that  EVM  was  substantially  more  optimistic  with  its 
schedule  predictions  than  ES.  This  test  illustrated  that  ES  is  a  more  accurate  predictor  of 
schedule  over  the  course  of  a  program.  Earned  Schedule  gives  more  accurate  signals  of 
what  the  final  SPI  value  will  be  over  the  entire  life  of  a  program,  lending  further  to  its 
credibility  as  a  schedule  analysis  tool  for  defense  acquisition  programs. 

Test  4:  Analysis  of  TSPI  Values 

Test  4  analyzes  different  To  Complete  Schedule  Performance  Index  (TSPI) 
values,  and  determines  at  which  program  completion  point  each  contract  achieves  that 
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value.  The  TSPI  is  the  SPI  efficiency  that  must  be  achieved  from  the  current  month  until 
the  estimated  project  completion  date  in  order  to  come  in  on  time.  The  literature 
suggests  that  any  program  where  the  TSPI  reaches  1 . 1  in  unrecoverable,  and  stands  no 
chance  of  meeting  its  schedule  objectives.  Of  the  64  programs  studied  in  this  research, 
zero  met  their  initial  estimated  completion  date.  Of  those  64  programs,  there  were  three 
that  never  had  a  TSPI  value  greater  than  1.10  despite  not  being  delivered  on  time.  Two 
of  those  three  programs  also  never  achieved  a  TSPI  value  of  greater  than  1.05:  again, 
neither  of  these  programs  were  delivered  to  the  customer  on  their  original  estimated 
completion  date.  All  64  of  the  programs  studied  had  a  TSPI  value  greater  than  1.01. 

TSPI  Greater  Than  1.10 

As  the  literature  suggests,  a  TSPI  value  equal  to  1.10  indicates  that  unless  an  SPI 
efficiency  equal  to,  or  greater  than  10%  more  than  the  planned  value  of  remaining  work 
on  a  program  is  completed,  the  program  will  not  be  delivered  on  time.  Figure  15  depicts 
the  program  completion  percentages  where  a  TSPI  value  greater  than  or  equal  to  1.10  is 
first  achieved.  The  mean  completion  percentage  at  which  a  program  first  achieved  a 
TSPI  value  of  1.10  or  greater  is  64.22%.  The  median  percentage  completion  point  is 
65.52%.  The  earliest  a  program  achieved  a  TSPI  value  greater  than  1.10  is  the  22.22% 
completion  point,  while  the  latest  was  94.12%. 
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TSPI  >  1.10 


■  Frequency 


Program  Completion  Percentage 


Figure  15:  TSPI  >  1.10 


While  it  is  helpful  to  have  an  understanding  at  the  64%  completion  point  that  an 
additional  10%  efficiency  is  required,  this  is  a  point  late  in  the  program  where  many 
decisions  have  been  made  and  many  dollars  expensed:  with  only  one-third  of  the 
program  remaining,  it  would  be  very  difficult  to  operate  with  such  efficiency.  With  that 
in  mind,  we  explored  if  lowering  that  standard  gave  an  earlier  warning  signal  that  a 
program  was  in  trouble. 

TSPI  Greater  Than  1.05 

We  chose  a  TSPI  greater  than  1.05  as  the  next  threshold  to  test.  A  histogram, 
depicted  in  Figure  16,  is  used  to  visually  depict  the  program  completion  percentages 
where  a  TSPI  value  greater  than  or  equal  to  1 .05  is  first  achieved.  The  mean  completion 
percentage  at  which  a  program  first  achieved  a  TSPI  value  of  1.05  or  greater  is  50.81%. 
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The  median  percentage  completion  point  is  50.39%.  The  earliest  a  program  achieved  a 
TSPI  value  greater  than  1.05  was  the  10.00%  completion  point,  while  the  latest  was 
88.24%. 


TSPI  >  1.05 


Program  Completion  Percentage 


Figure  16:  TSPI  >  1.05 


Of  the  64  programs  analyzed,  the  TSPI  value  reached  1.10  on  61  of  the  program, 
while  the  TSPI  value  reached  1.05  of  62  of  the  programs.  This  indicated  that  there  is 
only  one  program  where  a  program  reaches  the  1.05  threshold  where  it  doesn’t  later  reach 
the  1.10  threshold.  Those  programs  are  likely  the  meet  the  TSPI  greater  than  1.10  value 
that  we  argue  is  too  high:  using  the  TSPI  greater  than  1.05  as  the  threshold  would  give 
the  program  an  earlier  indication  that  action  must  be  taken.  There  is  enhanced  value  in 
using  the  TSPI  greater  than  1.05  metric  in  lieu  of  the  currently  accepted  TSPI  greater  than 
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1.10  value,  but  it  still  fails  to  detect  a  problem  until  the  program  is  halfway  complete. 
Our  final  test  observed  the  advantages  gained  by  using  a  TSPI  greater  than  1.01. 


TSPI  Greater  Than  1.01 

The  final  threshold  we  tested  measured  the  first  time  the  TSPI  value  reached  1.01. 
A  histogram,  depicted  in  Figure  17,  is  used  to  visually  depict  the  program  completion 
percentages  where  a  TSPI  value  greater  than  or  equal  to  1.01  is  first  achieved.  The  mean 
completion  percentage  at  which  a  program  first  achieved  a  TSPI  value  of  1.01  or  greater 
is  24.24%.  The  median  percentage  completion  point  is  18.18%.  The  earliest  a  program 
achieved  a  TSPI  value  greater  than  1.01  was  the  2.94%  completion  point,  while  the  latest 
was  75.00%.  Of  all  64  programs  studied,  only  6  achieved  a  TSPI  value  greater  than  1.01 
later  than  the  50%  completion  point. 


TSPI  >  1.01 


Program  Completion  Percentage 


■  Frequency 


Figure  17:  TSPI  >  1.01 
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Of  the  64  programs  studied,  each  one  reached  a  TSPI  value  of  greater  than  1.01. 
There  were  only  three  instances  where  the  program  didn’t  reach  a  TSPI  value  of  1.10. 

We  see  that  on  61  of  64  occasions  (95.31%),  a  program  that  reaches  a  TSPI  value  of  1.01 
will  eventually  reach  a  TSPI  value  of  1.10.  Using  the  TSPI  value  greater  than  1.01  as  the 
“indicator”  that  a  program  is  in  trouble  will  not  falsely  identify  programs  that  are  in 
trouble,  but  rather,  will  provide  the  program  manager  additional  time  (and  resources)  to 
make  adjustments  to  the  program  to  allow  for  an  on-time  delivery. 

There  was  one  drawback  to  this  approach:  there  were  several  occasions  where  a 
TSPI  would  reach  a  certain  threshold,  often  by  only  tenths  or  hundredths  a  point.  The 
next  reporting  period  would  then  reduce  that  TSPI  value  below  the  aforementioned 
threshold.  To  account  for  such  circumstances,  we  studied  the  TSPI  values  at  six 
thresholds  throughout  each  of  the  programs:  20%,  40%,  50%,  60%,  80%,  and  90%.  The 
TSPI  for  each  contract  at  each  of  these  thresholds  was  examined,  and  if  it  was  equal  to,  or 
greater  than  the  TSPI  value,  it  was  counted.  The  results  can  be  seen  in  Table  12  below: 
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Table  12:  TSPI  Comparison  Over  Time 


20% 

40% 

50% 

60% 

80% 

90% 

TSPI  >  1.01 

35 

46 

51 

51 

62 

57 

TSPI  >  1.05 

9 

21 

31 

40 

53 

55 

TSPI  >  1.10 

3 

9 

14 

25 

45 

51 

The  data  is  also  presented  graphically  in  Figure  18  below: 


Figure  18:  TSPI  Comparison  Over  Time 


From  this  analysis,  we  determined  that  there  is  a  significant  disparity  in  the 
detection  capability  when  a  TSPI  greater  than  1.10  is  used  rather  than  TSPI  greater  than 
1.05  or  TSPI  greater  than  1.01,  especially  over  the  first  50%  of  the  program.  As  the 
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program  reaches  the  60%  completion  point,  the  relative  improvement  of  TSPI  greater 
than  1.01  is  somewhat  reduced,  but  it  is  still  significantly  more  accurate  than  the  other 
thresholds.  That  margin  declines  further  at  the  80%  program  completion  point,  and  the 
difference  between  using  a  TSPI  greater  than  1.01,  1.05  or  1.10  is  negligible  at  the  90% 
program  completion  point. 

One  potential  critique  to  using  such  a  low  value  for  the  TSPI  is  that  one  poor 
month  early  in  the  program  will  result  in  a  very  early  detection.  That,  however,  is  not 
necessarily  the  case:  the  TSPI  takes  into  account  the  anticipated  remaining  duration  of  a 
program,  and  accounts  for  the  program’s  ability  to  make  up  lost  efficiency  over  that 
period.  Using  an  earlier  value  for  detection  also  prevents  forcing  the  program  manager  to 
make  drastic  changes  to  the  program. 

Test  5:  Analysis  of  SV(t)  vs.  SV($)  Divergence  Point 

The  final  test  conducted  on  the  data  set  studied  the  schedule  variance  (SV(t)  and 
SV($))  for  each  contract.  For  each  contract,  the  SV(t)  and  SV($)  were  calculated  and 
graphed  on  a  scatter  plot.  The  scatter  plot  had  two  vertical  axes:  one  measured  in  dollars 
to  accommodate  the  EV  calculation  and  one  measured  in  time  to  accommodate  the  ES 
calculation.  The  ES  literature  identified  the  behavior  of  the  two  schedule  variance 
methodologies:  this  research  attempted  to  identify  at  what  percentage  completion  point  in 
a  program  that  the  SV(t)  and  SV($)  diverge.  The  scatter  plots  illustrated  how  the  SV(t) 
and  SV($)  changed  over  the  course  of  a  program.  Figure  19  exhibits  the  scatter  plot  for  a 
typical  program,  in  this  case  the  BIB  Propulsion  Lot  1  contract.  See  Appendix  C  for 
SV(t)  vs.  SV($)  scatter  plots  for  all  programs. 
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BIB  Propulsion  Lot  1  SV(t)  vs.  SV{$) 


— »-SV{$) 


— SV{t) 


Figure  19:  BIB  Propulsion  Lot  1  SV(t)  vs.  SV($) 


For  this  analysis,  the  divergence  point  on  each  of  the  scatter  plots  is  identified, 
and  put  in  terms  of  program  percentage  completion.  The  divergence  occurs  when  the 
SV($)is  no  longer  growing  and  begins  its  regression  towards  zero.  In  Figure  19,  the 
divergence  is  determined  to  occur  during  the  5th  reporting  period,  which  was  the  45.45% 
completion  point  of  the  program.  The  final  reporting  period  for  this  program  specifies 
that  the  SV($)  is  -$1.4M,  while  the  SV(t)  is  -2.33  quarters,  or  7  months  behind  schedule. 
Were  the  final  CPR  data  made  available  by  the  contractor,  we  surmise  that  the  SV($) 
would  have  regressed  closer  to  zero,  while  the  SV(t)  would  have  reamined  constant  or 
continued  to  decrease. 

The  divergence  points,  in  terms  of  percentage,  were  calculated  for  each  of  the 
programs.  Of  the  64  programs  studied  for  this  research,  there  were  three  where  there  was 
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no  discemable  divergence  point:  ALCM  Interface,  F-15  Airframe  Development  and  F-15 


Engine  Lot  7  Production.  For  our  analysis,  these  three  data  points  were  removed.  The 
mean  divergence  point  was  50.62%  program  completion,  while  the  median  divergence 
point  was  50.00%  completion.  The  divergence  point  occurred  as  early  in  the  program  as 
the  10.34%  completion  point  (C-17  Performance  Enhancement)  and  as  late  as  the  86.67% 
completion  point  (F-15  ALQ  Update  3).  A  histogram  showing  the  distribution  of  the 
percentage  completion  points  is  presented  in  Figure  20  below: 


Comparison  of  SV(t)  vs.  SV{$)  Divergence 

Point 


Percentage  Completion  Point 


Figure  20:  Comparison  of  SV(t)  vs.  SV($)  Divergence  Point 


The  results  of  this  test  indicate  that  at  approximately  the  50%  program  completion 
point,  the  SV($)  metric  provides  little  value  to  the  decision  maker.  There  are  two 
advantages  to  using  the  Earned  Schedule  metric  as  a  measure  of  schedule  variance.  The 
first,  outlined  extensively  in  the  literature,  is  that  Earned  Schedule  puts  SV  values  in 
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terms  of  time,  rather  than  in  terms  of  dollars.  The  second  advantage,  detailed  in  this 
research,  is  that  Earned  Schedule  provides  more  accurate  and  realistic  schedule  variance 
indicators  to  the  program  manager  over  the  second  half  of  a  program. 

Application  to  the  Critical  Path 

One  criticism  to  Earned  Schedule  has  been  whether  it  is  appplicable  when  using 
the  Critical  Path  Method  (CPM).  The  literature,  predominantly  from  Walt  Lipke’s  2006 
paper  titled  Applying  Earned  Schedule  to  Critical  Path  Analysis  and  More,  suggests  that 
Earned  Schedule  is  applicable  to  the  Critical  Path.  This  research  investigated  Earned 
Scehdule’s  applicability  to  the  Critical  Path  Method  on  the  C130J  Block  7.0  Upgrade 
program.  The  results  show  a  fundamental  disconnect  between  what  Earned  Value  data  is 
collected  and  the  Critical  Path  data  used  by  the  program  office.  Our  example  produced 
earned  value  data  no  deeper  than  the  Work  Breakdown  Schedule  (WBS)  level  3  (ex: 

WBS  Element  1.2.3).  The  Critical  Path  data  is  collected  much  deeper,  as  detailed  as 
WBS  level  7  (ex:  WBS  Element  1.2.3.4.5.6.7).  This  disconnect  prevented  us  from 
conducting  a  detailed  analysis.  This  does  not  necessarily  suggest  that  Earned  Schedule  is 
inapplicable  to  the  Critical  Path  Method,  but  conclusive  research  demands  more  detailed 
Earned  Value  data. 

Conclusion 

The  result  of  our  qualitative  research  indicate  that  Earned  Schedule  has  not  been 
widely  implemented  in  DoD  acquisition  programs  since  its  introduction  a  decade  ago. 
While  some  programs  use  it  as  a  crosscheck  tool,  Earned  Value  Management  continues  to 
be  the  primary  schedule  analysis  tool  for  all  defense  acquisiton  programs.  The  results  of 
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our  quantitative  tests  conclude  that  there  is  a  stastistically  significant  difference  between 
the  results  of  Earned  Value  Management  and  Earned  Schedule,  and  that  Earned  Schedule 
provides  more  timely  and  more  accurate  schedule  predictions  than  traditional  DoD 
methods.  The  next  chapter  discusses  the  implications  of  these  findings  and 
recommendations  for  future  implementation  of  Earned  Schedule. 
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V.  Conclusions  and  Recommendations 


Introduction 

Earned  Value  Management  has  long  been  the  preferred  tool  for  program 
management  and  cost/schedule  forecasting  on  United  States  Department  of  Defense 
(DoD)  programs.  In  2003,  Walt  Lipke,  in  his  seminal  paper  Schedule  is  Different, 
identified  EVM’s  two  major  shortcomings  as  schedule  analysis  tool:  it  quantifies 
schedule  overages  in  terms  of  dollars,  rather  than  in  terms  of  time,  and  it  is  effectively 
useless  over  the  final  third  of  a  program.  Earned  Schedule  uses  the  same  data  collected 
for  Earned  Value  Management  calculations,  but  focuses  on  giving  the  program  manager 
more  useful  information  that  overcomes  the  limitations  of  EVM. 

Our  research  concluded  that  there  is  a  statistical  difference  between  the  results  of 
Earned  Schedule  and  Earned  Value  Management  tests  conducted  on  Major  Defense 
Acquisition  Programs  from  Aeronautical  Systems  Center.  Additionally,  we  answered  our 
original  research  questions  first  introduced  in  Chapter  1. 

Research  Questions  Answered 

Our  first  question  was,  “To  what  extent  is  Earned  Schedule  currently  utilized  in 
Air  Force  ACAT  I  acquisition  programs?”  We  determined  that  no  System  Program 
Offices  (SPO),  regardless  of  product  center,  uses  Earned  Schedule  as  their  primary 
schedule  analysis  tool.  Of  the  seventeen  program  offices  that  responded  to  our  inquiry, 
six  program  offices  use  Earned  Schedule  in  some  capacity.  Those  who  use  Earned 
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Schedule  use  it  exclusively  as  a  secondary  analysis  tool,  and  as  a  cross-check  to  the 
results  obtained  from  the  Earned  Value  Management  analysis. 

Our  next  question  asked,  “Does  Earned  Schedule  provide  more  accurate  schedule 
predictions  than  traditional  DoD  methods?”  We  determined  that  Earned  Schedule  was  a 
more  accurate  predictor  of  schedule  than  Earned  Value  Management.  At  the  50% 
completion  point,  the  average  SPI($)  metric  was  .958,  while  the  average  SPI(t)  metric 
was  .934.  Earned  Schedule  showed  its  value  as  a  more  accurate  schedule  predictor  even 
later  in  the  program:  at  the  90%  completion  point,  the  average  SPI($)  was  .974,  while  the 
SPI(t)  for  the  same  data  was  .923.  Another  measure  of  how  ES  is  more  accurate  than 
EVM  was  in  how  frequently  each  suggested  the  program  was  “in  trouble.”  We 
determined  that  any  SPI  value  below  .90  meant  the  program  was  in  trouble.  Of  the  1,087 
points  we  studied,  EVM  indicated  only  135  points  (12.42%)  were  below  .90,  while  ES 
disclosed  that  220  points  (20.24%)  were  determined  to  be  in  trouble.  Another  measure 
compared  how  ES  predicted  problems  over  time  compared  to  EVM:  at  the  60% 
completion  point,  ES  predicted  14  programs  were  in  trouble,  while  EVM  predicted  5 
programs  were  in  trouble.  More  telling,  at  the  90%  completion  point,  ES  indicated  20 
programs  were  in  trouble  while  EVM  predicted  only  1  would  come  in  behind  schedule. 

A  further  measure  we  used  to  determine  which  method  was  more  accurate  was  the 
frequency  of  which  method  was  more  optimistic.  The  more  optimistic  measure  was 
determined  to  be  the  higher  of  the  two  SPI  values:  the  higher  SPI  value  indicates  the 
program  is  more  likely  to  come  in  on  time.  Of  our  1,087  data  points,  Earned  Value 
Management  was  more  optimistic  for  646  (59.43%)  of  the  data  points,  while  ES  was 
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more  optimistic  for  381  (35.05%)  of  the  data  points.  Earned  Schedule  proved  to  be  more 
accurate  later  in  the  life  of  a  program:  EVM  was  more  optimistic  38  times,  compared  to 
3 1  programs,  at  the  50%  completion  point.  However,  at  the  90%  completion  point, 

EYM  was  more  optimistic  on  59  programs,  while  Earned  Schedule  was  more  optimistic 
on  only  5.  Our  series  of  tests  confirmed  that  Earned  Schedule  is  a  more  accurate 
schedule  predictor  than  the  Earned  Value  Management  technique  currently  employed  by 
the  Department  of  Defense  on  Major  Defense  Acquisition  Programs. 

Our  final  question  was,  “Does  Earned  Schedule  provide  more  timely  schedule 
predictions  than  traditional  DoD  methods?”  We  concluded  that  Earned  Schedule  was  a 
more  timely  predictor  of  schedule  overages  than  Earned  Value  Management.  While  on 
average,  Earned  Value  Management  first  detected  a  problem  in  a  program  (SPI  value 
below  .90)  at  the  17.89%  completion  point  as  opposed  to  Earned  Schedule  detected  at  the 
29.89%  completion  point,  this  failed  to  account  for  the  9  programs  where  EVM  neglected 
to  identify  a  problem  existed  where  ES  did.  Further,  at  the  20%  completion  point,  EVM 
only  identified  12  programs  that  were  “in  trouble,”  while  ES  detected  issues  with  20 
programs.  A  comparison  of  the  SPI  Closer  to  the  Final  Schedule  in  Figure  14  concluded 
that  Earned  Schedule  was  closer  to  the  final  result  more  frequently  than  Earned  Value 
Management  as  early  as  the  20%  program  completion  point.  The  further  the  program 
progressed,  the  more  frequently  the  SPI(t)  was  closer  to  the  final  schedule.  Finally,  it 
was  determined  that  the  SV(t)  and  SV($)  had  an  average  divergence  point  of  50.62%. 

This  conclusion  dictates  that  the  SV($)  metric  is  comparatively  useless  over  the  final  half 
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of  the  program,  while  Earned  Schedule  provides  meaningful  information  over  the  entire 
life  of  a  program. 

An  ancillary  finding  for  our  research  came  when  researching  the  To  Complete 
Schedule  Performance  Index  (TSPI).  The  literature  suggests  that  when  the  TSPI  reaches 
1.10,  the  program  manager  must  take  action  to  improve  the  schedule  performance  in 
order  to  deliver  the  system  on  time.  Our  research  concluded  that  using  a  TSPI  of  1.10 
didn’t  alert  the  program  manager,  on  average,  until  the  64.22%  completion  point  that  a 
program  was  experiencing  issues.  Rather,  using  a  TSPI  threshold  of  greater  than  1.01 
gave  the  program  manager  an  indication  at  the  24.24%  completion  point  that  program 
was  in  trouble.  Of  the  64  programs  studied,  63  programs  achieved  a  TSPI  value  greater 
than  1.10,  while  60  programs  reached  a  TSPI  greater  than  1.01.  Therefore,  using  a  TSPI 
of  greater  than  1.01  as  an  indicator  of  “trouble,”  would  only  designate  3  programs  with 
issues  that  a  TSPI  of  1.10  failed  to,  and  would  alert  the  program  manager,  on  average, 
40%  sooner  that  action  must  be  taken  to  improve  the  program’s  performance. 

Recommendations 

We  recommend  that  program  offices  implement  Earned  Schedule  as,  at  the  very 
least,  a  supplementary  tool  for  their  schedule  analysis.  Earned  Schedule  has  been  on  the 
fringe  of  gaining  acceptance  in  the  defense  acquisition  community  for  several  years,  but 
we  recommend  escalating  its  usage  in  the  program  office.  One  shortcoming  to  this 
research  is  the  inability  to  map  the  Earned  Schedule  data  to  the  critical  path,  but  we 
consider  Earned  Schedule  to  be  a  strong  tool  for  schedule  prediction  at  the 
summary/contract  level. 
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Another  recommendation  is  to  include  Earned  Schedule  in  the  Defense 


Acquisition  University  (DAU)  curriculum.  Several  analysts  noted  that  shrinking 
workforces  are  creating  work  environments  where  on-the-job  training  (OJT)  is  less 
viable.  Introducing  Earned  Schedule  into  the  DAU  curriculum  would  allow  analysts  in 
the  program  offices  the  opportunity  to  learn  the  basics  of  Earned  Schedule,  how  to 
perform  the  analysis  and  calculations,  and  give  them  insight  into  what  these  new  metrics 
offer  their  program  managers. 

Future  Research 

Our  research  was  conducted  on  the  system  platforms  from  a  single  product  center, 
and  was  accomplished  at  a  general  level.  We  recommend  conducting  parallel  research  on 
the  other  two  major  system  platforms:  Electronic  systems,  as  well  as  Space  and  Missiles 
systems.  There  are  several  differences  between  the  particular  platforms  that  could  lead 
to  dissimilar  results  than  what  we  found:  the  size  of  the  programs,  the  length  of  the 
acquisition  process,  the  integration  of  Commercial  Off  The  Shelf  (COTS)  products,  and 
the  level  of  security  classification.  Another  future  opportunity  builds  off  of  this  research: 
a  comparison  of  the  different  classes  of  aeronautical  platforms.  Researching  if  there  is  a 
significant  difference  between  implementing  Earned  Schedule  versus  Earned  Value 
Management  on  Fighter  Aircraft,  Bomber  Aircraft,  Cargo  Aircraft,  Unmanned  Aircraft, 
and  if  the  data  becomes  available,  Rotary  Wing  Aircraft.  Again,  there  are  differences 
between  these  platforms  based  on  technological  sophistication  and  number  of  units 
produced.  An  additional  proposal  for  future  research  would  be  very  similar  to  the 
comparison  of  different  classes  of  aeronautical  platforms:  a  comparison  of  how  well 
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Earned  Schedule  performs  as  a  schedule  predictor  versus  Earned  Value  Management  for 
the  difference  prime  contractors  used  by  the  DoD. 

There  was  one  avenue  we  hoped  to  pursue  but  time  prevented  us.  We  noticed, 
anecdotally,  that  there  seemed  to  be  an  inflection  point  where  SPI($)  and  SPI(t)  diverted: 
SPI($)  moving  towards  1.0,  and  SPI(t)  away  from  it.  We  recommend  researching  at  what 
program  completion  point  that  inflection  occurs,  as  well  as  researching  a  potential  root 
cause  for  such  an  inflection. 
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DD  FORM  2734/5,  MAR  05 

LOCAL  REPRODUCTION  AUTHORIZED. 

CLASSIRCATION  (When  Filled  In) 

Figure  25:  Example  Format  5 
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Appendix  B 


A-10  Production  Option  1  SPI(t)  vs.  SPI($) 


♦  SPI{t) 
■  SPI{$) 


Figure  26:  A-10  Production  Option  1  SPI(t)  vs.  SPI($) 
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Figure  27:  A-10  Production  Option  3/4  SPI(t)  vs.  SPI($) 
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A-10  Production  Option  5  SPI(t)  vs.  SPI($) 


Reporting  Period  {Months) 


♦  SPI{T) 
■  SPI($) 


Figure  28:  A-10  Production  Option  5  SPI(t)  vs.  SPI($) 
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A-10  Production  Option  6  SPI(t)  vs.  SPI($) 
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■  SPI(S) 


Figure  29:  A-10  Production  Option  6  SPI(t)  vs.  SPI($) 
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Figure  30:  A-10  Production  Option  7  SPI(t)  vs.  SPI($) 
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F-15  Airframe  Development  SPI(t)  vs.  SPI($) 
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■  SPII($) 


Figure  31:  F-15  Airframe  Development  SPI(t)  vs.  SPI($) 
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BIB  Offensive  Avionics  Lot  1  SPI(t)  vs.  SPI($) 


Reporting  Period  {Quarters) 


♦  SPI{T) 
■  SPI($) 


Figure  32:  BIB  Offensive  Avionics  Lot  1  SPI(t)  vs.  SPI($) 
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BIB  Defensive  Avionics  Lot  1  SPI(t)  vs.  SPI($) 


Reporting  Period  {Quarters) 


Figure  33:  BIB  Defensive  Avionics  Lot  1  SPI(t)  vs.  SPI($) 


♦  SPI(t) 
■  SPI($) 
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BIB  Offensive  Avionics  Lot  2  SPI(t)  vs.  SPI($) 


Reporting  Period  {Quarters) 


Figure  34:  BIB  Offensive  Avionics  Lot  2  SPI(t)  vs.  SPI($) 


♦  SPI(t) 
■  SPI($) 
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BIB  Defensive  Avionics  Lot  2  SPI(t)  vs.  SPI($) 


♦  SPI{T) 
■  SPI($) 


Figure  35:  BIB  Defensive  Avionics  Lot  2  SPI(t)  vs.  SPI($) 
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BIB  Offensive  Avionics  Lot  3  SPI(t)  vs.  SPI($) 


Reporting  Period  {Quarters) 


Figure  36:  BIB  Offensive  Avionics  Lot  3  SPI(t)  vs.  SPI($) 


♦  SPI{T) 
■  SPI($) 
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o_ 

1-0 


BIB  Defensive  Avionics  Lot  3  SPI(t)  vs.  SPI($) 
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Figure  37:  BIB  Defensive  Avionics  Lot  3  SPI(t)  vs.  SPI($) 
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BIB  Offensive  Avionics  Lot  4  SPI(t)  vs.  SPI($) 


Reporting  Period  {Quarters) 


Figure  38:  BIB  Offensive  Avionics  Lot  4  SPI(t)  vs.  SPI($) 


♦  SPI(t) 
■  SPI($) 
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BIB  Defensive  Avionics  Lot  4  SPI(t)  vs.  SPI($) 


Reporting  Period  {Quarters) 


♦  SPI(t) 
■  SPI($) 


Figure  39:  BIB  Defensive  Avionics  Lot  4  SPI(t)  vs.  SPI($) 
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BIB  Offensive  Avionics  Lot  5  SPI(t)  vs.  SPI ($) 


Reporting  Period  {Quarters) 


♦  SPl(t) 
■  SPI($) 


Figure  40:  BIB  Offensive  Avionics  Lot  5  SPI(t)  vs.  SPI($) 
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BIB  Defensive  Avionics  Lot  5  SPI(t)  vs.  SPI ($) 


Reporting  Period  {Quarters) 


♦  SPI{T) 
■  SPI($) 


Figure  41:  BIB  Defensive  Avionics  Lot  5  SPI(t)  vs.  SPI($) 
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F-15  FY78  Production  &  Support  SPI(t)  vs.  SPI($) 


Figure  42:  F-15  FY78  Production  &  Support  SPI(t)  vs.  SPI($) 


♦  SPI{T) 

■  spi(S) 
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F-15  Engine  Production  Lot  8  SPI(t)  vs.  SPI ($) 


Reporting  Period  {Quarters) 


♦  SPI{T) 
■  SPI($) 


Figure  43:  F-15  Engine  Production  Lot  8  SPI(t)  vs.  SPI($) 


113 


BIB  Propulsion  Lot  1  SPI(t)  vs.  SP!($) 


Reporting  Period  {in  Quarters) 


Figure  44:  BIB  Propulsion  Lot  1  SPI(t)  vs.  SPI($) 


♦  SPI{t) 
■  SPI{$) 
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BIB  Propulsion  Lot  2  SPI(t)  vs.  SPi($) 


Reporting  Period  {Quarters) 


Figure  45:  BIB  Propulsion  Lot  2  SPI(t)  vs.  SPI($) 


♦  SPl(t) 

■  spi(S) 
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BIB  Airframe  Rockwell  Production  SPI(t)  vs.  SPI($) 
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Figure  46:  BIB  Airframe  Rockwell  Production  SPI(t)  vs.  SPI($) 
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BIB  Airframe  Production  SPI(t)  vs.  SPI($) 


Reporting  Period  {Quarters) 


Figure  47:  BIB  Airframe  Production  SPI(t)  vs.  SPI($) 


♦  SPI(t) 
■  SPI($) 
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F-16  FY  85  Multiyear  Production  SPI(t)  vs.  SPI ($) 


Reporting  Period  {Quarters) 


♦  SPl(t) 
■  SPI($) 


Figure  48:  F-16  FY  85  Multiyear  Production  SPI(t)  vs.  SPI($) 
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Figure  49:  F-15  MSIP  Phase  2 
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F-15  Engine  Lot  7  Production  SPI(t)  vs.  SPI($) 


Reporting  Period  {Quarters) 


Figure  50:  F-15  Engine  Lot  7  Production  SPI(t)  vs.  SPI($) 


♦  SPl(t) 

■  spi(S) 
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F-15  ALQ  Update  Lot  3  SPI(t)  vs.  SPI($) 


Reporting  Period  {Quarters) 


Figure  51:  F-15  ALQ  Update  Lot  3  SPI(t)  vs.  SPI($) 


♦  SPI(t) 

■  spi(S) 
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T-46  Engine  Development  SPI(t)  vs.  SPI($) 


Reporting  Period  {Quarters) 


Figure  52:  T-46  Engine  Development  SPI(t)  vs.  SPI($) 


♦  SPI(t) 

■  spi(S) 
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T-46  Airframe  Development  SPI(t)  vs.  SPI($) 


Reporting  Period  {Quarters) 


♦  SPl(t) 
■  SPI($) 


Figure  53:  T-46  Airframe  Development  SPI(t)  vs.  SPI($) 
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F-15  AMSIP  Production  SPI(t)  vs.  SPI($) 


Reporting  Period  {Quarters) 


Figure  54:  F-15  AMSIP  Production  SPI(t)  vs.  SPI($) 


♦  SPl(t) 

■  spi(S) 
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EF-111ATJS  SPI(t)  vs.  SPI($) 
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Figure  55:  EF-111A  TJS  SPI(t)  vs.  SPI($) 


♦  SPl(t) 

■  spi(S) 
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EF-111A  TJS  Production  SPI(t)  vs.  SPI($) 


0  2  4  6  8  10  12  14 


Reporting  Period  {Quarters) 


♦  SPI(t) 
■  SPI($) 


Figure  56:  EF-111A  TJS  Production  SPI(t)  vs.  SPI($) 
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IR  Maverick  Development  SPI(t)  vs.  SPI($) 


♦  SPI(t) 
■  SPI($) 


Figure  57:  IR  Maverick  Development  SPI(t)  vs.  SPI($) 
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Maverick  Production  Segment  1  SPI(t)  vs.  SPI($) 
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Figure  58:  Maverick  Production  Segment  1  SPI(t)  vs.  SPI($) 


♦  SPI(t) 

■  spi(S) 
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Maverick  Production  SPI(t)  vs.  SPI{$) 


Reporting  Period  {Quarters) 


♦  SPl(t) 
■  SPI($) 


Figure  59:  Maverick  Production  SPI(t)  vs.  SPI($) 
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Maverick  Production  Lot  2  SPI(t)  vs.  SPI($) 


Reporting  Period  {Quarters) 


Figure  60:  Maverick  Production  Lot  2  SPI(t)  vs.  SPI($) 


♦  SPI(t) 
■  SPI($) 
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AWACS  Group  A  SPI(t)  vs.  SPI($) 


♦  SPl(t) 
■  SPI($) 


Figure  61:  AWACS  Group  A  SPI(t)  vs.  SPI($) 
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AWACS  Group  B  SPI(t)  vs.  SPI($) 
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Figure  62:  AWACS  Group  B  SPI(t)  vs.  SPI($) 
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ALCM  Engine  DevelopmentSPI(t)  vs.  SPI ($) 
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Figure  63:  ALCM  Engine  Development  SPI(t)  vs.  SPI($) 


♦  SPl(t) 
■  SPI($) 


133 


ALCM  Airframe  DevelopmentSPI(t)  vs.  SPI ($) 


Reporting  Period  {Quarters) 


Figure  64:  ALCM  Airframe  Development  SPI(t)  vs.  SPI($) 


♦  SPl(t) 
■  SPI($) 
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ALCM  Airframe  Development  Lot  2  SPI(t)  vs.  SPI($) 


Reporting  Period  {Quarters) 


♦  SPl(t) 
■  SPI($) 


Figure  65:  ALCM  Airframe  Development  Lot  2  SPI(t)  vs.  SPI($) 
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ALCM  FY80  Air  Vehicle  Production  SPI(t)  vs.  SPI($) 
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Figure  66:  ALCM  FY80  Air  Vehicle  Production  SPI(t)  vs.  SPI($) 


♦  SPI(t) 
■  SPI($) 
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ALCM  FY81  Air  Vehicle  Production  SPI(t)  vs.  SPI($) 


Reporting  Period  {Quarters) 


Figure  67:  ALCM  FY81  Air  Vehicle  Production  SPI(t)  vs.  SPI($) 


♦  SPI(t) 
■  SPI($) 
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ALCM  Guidance  DevelopmentSPI(t)  vs.  SPI ($) 
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Figure  68:  ALCM  Guidance  Development  SPI(t)  vs.  SPI($) 
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ALCM  Land  Attack  SPI(t)  vs.  SPI($) 


Reporting  Period  {Quarters) 


Figure  69:  ALCM  Land  Attack  SPI(t)  vs.  SPI($) 


♦  SPl(t) 
■  SPI($) 
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ALCM  Interface  SPI(t)  vs.  SPI($) 


Reporting  Period  {Quarters) 


♦  SPl(t) 
■  SPI($) 


Figure  70:  ALCM  Interface  SPI(t)  vs.  SPI($) 
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B-2  RMD  Production  SPI(t)  vs.  SPI($) 


Reporting  Period  {Quarters) 


Figure  71:  B-2  RMD  Production  SPI(t)  vs.  SPI($) 


♦  SPl(t) 
■  SPI($) 
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Figure  72:  B-2  RMD  SDD  SPI(t)  vs.  SPI($) 
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F-16  FY91  Production 


Reporting  Period  {Quarters) 


♦  SPl(t) 
■  SPI($) 


Figure  73:  F-16  FY91  Production 
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Figure  74:  SRAM  II  SPI(t)  vs.  SPI($) 
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F-16  FY85  APG86  Production  SPI(t)  vs.  SPI($) 


Reporting  Period  {Quarters) 


Figure  75:  F-16  FY85  APG86  Production  SPI(t)  vs.  SPI($) 


♦  SPl(t) 

■  spi(S) 
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F-15  Engine  Lot  5  Production  SPI(t)  vs.  SPI ($) 


Reporting  Period  {Quarters) 


Figure  76:  F-15  Engine  Lot  5  Production  SPI(t)  vs.  SPI($) 


♦  SPl(t) 

■  spi(S) 
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F-1G  Airframe  DevelopmentSPI(t)  vs.  SPI ($) 
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Figure  77:  F-16  Airframe  Development  SPI(t)  vs.  SPI($) 
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C-17  FY88  Production 


♦  SPl(t) 
■  SPI($) 


Figure  78:  C-17  FY88  Production 
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C-17  RDT&E  &  Production  Lot  1  SPI(t)  vs.  SPI($) 


Figure  79:  C-17  RDT&E  &  Production  Lot  1  SPI(t)  vs.  SPI($) 


♦  SPl(t) 
■  SPI($) 


149 


C-17  Production  Lot  2  SPI(t)  vs.  SPI($) 


Reporting  Period  {Quarters) 


♦  SPl(t) 
■  SPI($) 


Figure  80:  C-17  Production  Lot  2  SPI(t)  vs.  SPI($) 
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C-17  Production  Lot  3  SPI(t)  vs.  SPI($) 


Reporting  Period  {Quarters) 


Figure  81:  C-17  Production  Lot  3  SPI(t)  vs.  SPI($) 


♦  SPl(t) 
■  SPI($) 
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C-17  Production  Lot  4  SPI(t)  vs.  SPI($) 


Reporting  Period  {Quarters) 


Figure  82:  C-17  Production  Lot  4  SPI(t)  vs.  SPI($) 


♦  SPl(t) 
■  SPI($) 
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C-17  Production  Lot  5  SPI(t)  vs.  SPI($) 


Reporting  Period  {Quarters) 


Figure  83:  C-17  Production  Lot  5  SPI(t)  vs.  SPI($) 


♦  SPl(t) 
■  SPI($) 
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C-17  Production  Lot  6  SPI(t)  vs.  SPI($) 


Reporting  Period  {Quarters) 


Figure  84:  C-17  Production  Lot  6  SPI(t)  vs.  SPI($) 


♦  SPl(t) 
■  SPI($) 
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C-17  Production  Lot  7  SPI(t)  vs.  SPI($) 


Reporting  Period  {Quarters) 


♦  SPl(t) 
■  SPI($) 


Figure  85:  C-17  Production  Lot  7  SPI(t)  vs.  SPI($) 
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C-17  Performance  Enhancement  SPI(t)  vs,  SPI ($) 
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Figure  86:  C-17  Performance  Enhancement  SPI(t)  vs.  SPI($) 
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C-17  Performance  Enhancement  Lot  2  SPI(t)  vs.  SPI($) 
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Figure  87:  C-17  Performance  Enhancement  Lot  2  SPI(t)  vs.  SPI($) 
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C-17  PEPI  II  SPI(t)  vs.  SPI($) 
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Figure  88:  C-17  PEPI  II  SPI(t)  vs.  SPI($) 
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C-130J  Block  7  Upgrade  SPI(t)  vs.  SPI($) 
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Figure  89:  C-130J  Block  7  Upgrade  SPI(t)  vs.  SPI($) 
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Appendix  C 


A-10  Production  Option  1  SV(t)  vs*  SV($) 
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Figure  90:  A-10  Production  Option  1  SV(t)  vs.  SV($) 
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A-10  Production  Option  3/4  SV(t)  vs.  SV{$) 
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Figure  91:  A-10  Production  Option  3/4  SV(t)  vs.  SV($) 
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A-10  Production  Option  5  SV(t)  vs.  SV{$) 
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Figure  92:  A-10  Production  Option  5  SV(t)  vs.  SV($) 
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A-10  Production  Option  6  SV(t)  vs.  SV{$) 
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Figure  93:  A-10  Production  Option  6  SV(t)  vs.  SV($) 
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A-10  Production  Option  7  SV{t)  vs.  SV{$) 
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Figure  94:  A-10  Production  Option  7  SV(t)  vs.  SV($) 
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F-15  Airframe  Development  SV(t)  vs.  SV($) 
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Figure  95:  F-15  Airframe  Development  SV(t)  vs.  SV($) 
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BIB  Offensive  Avionics  Lot  1  SV(t)  vs.  SV($) 
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Figure  96:  BIB  Offensive  Avionics  Lot  1  SV(t)  vs.  SV($) 
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BIB  Defensive  Avionics  Lot  1  SV(t)  vs.  SV($) 
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Figure  97:  BIB  Defensive  Avionics  Lot  1  SV(t)  vs.  SV($) 
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Figure  98:  BIB  Offensive  Avionics  Lot  2  SV(t)  vs.  SV($) 


168 


BIB  Defensive  Avionics  Lot  2  SV(t)  vs.  SV($) 
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Figure  99:  BIB  Defensive  Avionics  Lot  2  SV(t)  vs.  SV($) 
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BIB  Offensive  Avionics  Lot  3  SV(t)  vs.  SV($) 
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Figure  100:  BIB  Offensive  Avionics  Lot  3  SV(t)  vs.  SV($) 
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BIB  Defensive  Avionics  Lot  3  SV(t)  vs.  SV($) 
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Figure  101:  BIB  Defensive  Avionics  Lot  3  SV(t)  vs.  SV($) 
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BIB  Offensive  Avionics  Lot  4  SV(t)  vs.  SV($) 
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Figure  102:  BIB  Offensive  Avionics  Lot  4  SV(t)  vs.  SV($) 
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BIB  Defensive  Avionics  Lot  4  SV(t)  vs.  SV($) 
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Figure  103:  BIB  Defensive  Avionics  Lot  4  SV(t)  vs.  SV($) 
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BIB  Offensive  Avionics  Lot  5  SV(t)  vs.  SV($) 
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Figure  104:  BIB  Offensive  Avionics  Lot  5  SV(t)  vs.  SV($) 
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BIB  Defensive  Avionics  Lot  5  SV(t)  vs.  SV($) 
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Figure  105:  BIB  Defensive  Avionics  Lot  5  SV(t)  vs.  SV($) 


175 


F-15  FY78  Production  &  Support  SV(t)  vs.  SV($) 
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Figure  106:  F-15  FY78  Production  &  Support  SV(t)  vs.  SV($) 
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F-15  Engine  Lot  8  Production  SV(t)  vs.  SV($) 
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Figure  107:  F-15  Engine  Lot  8  Production  SV(t)  vs.  SV($) 
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BIB  Propulsion  Lot  1  SV(t)  vs.  SV{$) 
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Figure  108:  BIB  Propulsion  Lot  1  SV(t)  vs.  SV($) 
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BIB  Propulsion  Lot  2  SV(t)  vs.  SV{$) 
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Figure  109:  BIB  Propulsion  Lot  2  SV(t)  vs.  SV($) 
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BIB  Airframe  Rockwell  SV(t)  vs.  SV{$) 
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Figure  110:  BIB  Airframe  Rockwell  SV(t)  vs.  SV($) 
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BIB  Airframe  SV(t)  vs.  SV($) 
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Figure  111:  BIB  Airframe  SV(t)  vs.  SY($) 
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F-16  FY85  Multiyear  SV(t)  vs.  SV($) 
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Figure  112:  F-16  FY85  Multiyear  SV(t)  vs.  SV($) 
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F-15  MSIP  Phase  2  SV(t)  vs.  SV($) 
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Figure  113:  F-15  MSIP  Phase  2  SV(t)  vs.  SV($) 
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F-15  Engine  Lot  7  Production  SV(t)  vs.  SV($) 
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Figure  114:  F-15  Engine  Lot  7  Production  SV(t)  vs.  SV($) 
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F-15  ALQ  Update  3  SV(t)  vs.  SV($) 
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Figure  115:  F-15  ALQ  Update  3  SV(t)  vs.  SV($) 
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T-46  Engine  Development  SV(t)  vs.  SV($) 
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Figure  116:  T-46  Engine  Development  SV(t)  vs.  SV($) 
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T-4G  Airframe  Development  SV(t)  vs.  SV($) 
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Figure  117:  T-46  Airframe  Development  SV(t)  vs.  SV($) 
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F-15  AMSIP  Production  SV(t)  vs.  SV($) 
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Figure  118:  F-15  AMSIP  Production  SV(t)  vs.  SY($) 
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Figure  119:  EF-111A  TJS  SV(t)  vs.  SV($) 
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EF-111A  Production  SV(t)  vs.  SV($) 
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Figure  120:  EF-111A  Production  SV(t)  vs.  SY($) 
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Maverick  Development  SV(t)  vs.  SV{$) 
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Figure  121:  Maverick  Development  SV(t)  vs.  SV($) 
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Maverick  Segment  1  SV(t)  vs.  SV{$) 
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Figure  122:  Maverick  Segment  1  SV(t)  vs.  SY($) 
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Maverick  Production  SV(t)  vs.  SV($) 
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Figure  123:  Maverick  Production  SV(t)  vs.  SV($) 
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Maverick  Production  Lot  2  SV(t)  vs.  SV($) 
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Figure  124:  Maverick  Production  Lot  2  SV(t)  vs.  SV($) 
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AWACS  Group  A  SV(t)  vs.  SV{$) 
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Figure  125:  AWACS  Group  A  SV(t)  vs.  SV($) 
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AWACS  Group  B  SV(t)  vs.  SV{$) 
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Figure  126:  AWACS  Group  B  SY(t)  vs.  SV($) 
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ALCM  Engine  Development  SV(t)  vs.  SV($) 
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Figure  127:  ALCM  Engine  Development  SV(t)  vs.  SV($) 
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ALCM  Airframe  DevelopmentSV(t)  vs.  SV($) 
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Figure  128:  ALCM  Airframe  Development 
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ALCM  Airframe  Development  Lot  2  SV(t)  vs.  SV($) 
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Figure  129:  ALCM  Airframe  Development  Lot  2  SV(t)  vs.  SV($) 
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Figure  130:  ALCM  FY80  Air  Vehicle  Production  SV(t)  vs.  SV($) 
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ALCM  FY81  Air  Vehicle  Production  SV(t)  vs.  SV($) 
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Figure  131:  ALCM  FY81  Air  Vehicle  Production  SV(t)  vs.  SV($) 
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Figure  132:  ALCM  Guidance  Development  SV(t)  vs.  SV($) 
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ALCM  Land  Attack  SV(t)  vs.  SV{$) 
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Figure  133:  ALCM  Land  Attack  SY(t)  vs.  SV($) 


203 


ALCM  Interface  SV(t)  vs.  SV($) 
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Figure  134:  ALCM  Interface  SV(t)  vs.  SV($) 
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B-2  RMD  Production  SV(t)  vs.  SV{$) 
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Figure  135:  B-2  RMD  Production  SV(t)  vs.  SY($) 
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B-2  RMD  SDD  SV(t)  vs.  SV($) 
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Figure  136:  B-2  RMD  SDD  SV(t)  vs.  SV($) 
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F-16  FY91  Production  SV(t)  vs.  SV{$) 
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Figure  137:  F-16  FY91  Production  SV(t)  vs.  SV($) 
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SRAM  II  SV(t)  vs.  SV($) 
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Figure  138:  SRAM  II  SV(t)  vs.  SV($) 
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F-1G  FY85  APG8G  Production  SV(t)  vs.  SV($) 
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Figure  139:  F-16  APG86  Production  SV(t)  vs.  SV($) 
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Figure  140:  F-15  Engine  Lot  5  Production  SV(t)  vs.  SV($) 
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F-1G  Airframe  DevelopmentSV(t)  vs.  SV($) 
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Figure  141:  F-16  Airframe  Development  SV(t)  vs.  SV($) 
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C-17  FY88  Production  SV(t)  vs.  SV($) 
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Figure  142:  C-17  FY88  Production  SV(t)  vs.  SV($) 
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C-17  RDTE  &  Lot  1  Production  SV(t)  vs. 

SV{$) 


SV{$) 

SV{t) 


Figure  143:  C-17  RDTE  &  Lot  1  Production  SV(t)  vs.  SV($) 
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C-17  Lot  2  Production  SV(t)  vs.  SV($) 
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Figure  144:  C-17  Lot  2  Production  SV(t)  vs.  SV($) 
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C-17  Lot  3  Production  SV(t)  vs.  SV{$) 
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Figure  145:  C-17  Lot  3  Production  SV(t)  vs.  SV($) 
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C-17  Lot  4  Production  SV(t)  vs.  SV{$) 
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Figure  146:  C-17  Lot  4  Production  SV(t)  vs.  SV($) 
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C-17  Lot  5  Production  SV(t)  vs.  SV($) 
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Figure  147:  C-17  Lot  5  Production  SV(t)  vs.  SV($) 
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C-17  Lot  6  Production  SV(t)  vs.  SV($) 
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Figure  148:  C-17  Lot  6  Production  SV(t)  vs.  SV($) 
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C-17  Lot  7  Production  SV(t)  vs.  SV($) 
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Figure  149:  C-17  Lot  7  Production  SV(t)  vs.  SV($) 
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Figure  150:  C-17  Performance  Enhancement  SV(t)  vs.  SV($) 
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C-17  Performance  Enhancement  Lot  2  SV(t)  vs.  SV($) 
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Figure  151:  C-17  Performance  Enhancement  Lot  2  SV(t)  vs.  SV($) 
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Figure  152:  C-17  PEPI II  SV(t)  vs.  SV($) 
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Figure  153:  C-130J  Weapon  System  SV(t)  vs.  SV($) 
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